Diagrams R Us

Screenshot 2015-11-10 22.00.24

Programming is a subset of system thinking and modeling. System models are often displayed in diagrams, but not always. Sometimes, a model may reflect a bi-directional flow, in which case, we may use an equation. The equals sign (=) in any equation is shorthand for two opposing arrows. I've collected four types of models in Pinterest. These are labeled FlowMixed, FlowRule, FlowData, and FlowControl. We really have only one type of flow since control flow is a type of data flow where the datum is a simple signal (such as a wave containing "squares", pulses, leading, or trailing edges). Still, control vs. data flow is a common distinction within the computer science literature. Here is a brief guide to the three major flow types: (1) control flow is like a relay race: a flow of control that is frequently in the form of a sequence. Food recipes or cartoons are good examples of control flow: 1, 2, 3, .... Branching occurs based on a decision or a comparison (e.g., in engineering, we have comparators, and humans or control systems use decision blocks). If the flow is all about decisions, we call it a decision tree. (2) data flow is more general since data are generally processed along the way. Most computers prior to World War II were analog, and thus, data flow. The ancient Antikythera mechanism was data flow. Mechanical clocks are data flow machines. The real world is saturated with data flow, which is why most science and engineering disciplines employ these types of models in physical systems. (3) Rules suggest "micro flows." If this then that. Often rules are good for modeling sensing, controlling, actuating in that order.  Do you like thinking in terms of diagrams? If you do, go into engineering or system modeling. It is a great way of interpreting the world around you.

Modeling as the New Rhetoric


I just Googled "rhetoric." The top search result defines rhetoric as "the art of effective or persuasive speaking or writing, especially the use of figures of speech and other compositional techniques." Rhetoric is one of those topics that is fundamental to our society, and one of three ancient arts of discourse.  But, something interesting about rhetoric has been going on for the last half-century, and modeling and simulation (M&S) is at the core of the excitement. A quick diversion to Syria and Palmyra with Wired Magazine's article entitled "A Jailed Activist's 3-D Models could save Syria History from ISIS." Bassel Khartabil created 3-D models of the ancient ruins of Palmyra and is currently jailed in Syria. There is a group of online "activists, archivists, and archaeologists" releasing 3D models under the name The New Palmyra Project. First, this is a welcome project and a great humanitarian cause. Second, New Palmyra is an example of how rhetoric has been changing in the digital age. Rhetoric is no longer limited to videos, photographs, and written texts. Models, in the form of models of geometry and dynamics, represent the new rhetorical force. If you want strength in your argument, you rely on models. The Climate Change 2014 Synthesis Report Summary for Policymakers is based on multiple models of climate. While data charts are interesting, it is what is hidden behind the data that is even more interesting: models of how climate changes, with its many effects (e.g., flooding, wind damage). Speaking as a member of the modeling and simulation discipline, we need to embrace modeling with a capital "M", meaning models of information, dynamics, and 3D models like those initiated by Bassel Khartabil.

Models of Étretat


Gustave Courbet painted the Cliffs of Étretat in 1870, but other artists also painted the cliffs (Vasily Polenov in 1874 and Claude Monet in 1883). Let's put on our modeling hats and dive in. But before we do, let me give you a sense of "model." A model is a representation of something else. In art, the word maquette (French for scale model) is frequently employed. In science and engineering, we say "scale model."  So, you could make a scale model of the cliffs with plaster of Paris or a 3D printer. You might also create a virtual 3D model with a program such as Maya or Blender. You could also create a conceptual model of the cliffs in the form of a concept map. A concept map is a visual network diagram that also goes by other names: mind map and semantic network. These types of models are information or knowledge models. But an even more critical part of analyzing this painting is to consider the painting as a system. A system is a large set of components that can be subdivided into other components. A systems thinking approach to viewing the Cliffs of Étretat usually involves a set of diagrams that capture how the system works. The cliffs were created through erosion. How, and when, did this erosion occur? The boats were built. How, and by whom? What was the process used in boat building in the late 19th century? The sky indicates a cloud pattern. What sort of weather process creates this pattern? How can a simulation model be constructed for the waves crashing on the beach? What is a dynamic, temporal model to re-create the sound of waves? Do the waves sound different in this location in France? What process was used by the artist to create the pigments? The paint is part of a larger system involving mining and chemistry. This type of thinking is similar to what Bonnie Pitman, Cassini Nazir, and I focused on with Models of X.  Thinking in models and systems reaches across disciplinary boundaries and so we need to create people networks, since being a a generalist requires coordination with specialists. Thinking of the world in terms of complex systems involving interacting components is critical to our society. Consider climate change and improving health by battling disease. Generalists, modelers, systems thinkers, and specialists need to work together. I'd like to think that Gustave would have approved.

What is Code?


One of the best public examples of coding that I've seen can be found at the Boston Museum of Science. There is a table and two people sit at either end. In the middle of the table, there is a vertical partition much like in the game of Battleship. Both people have an identical set of blocks of different shapes, sizes, and colors. One person issues verbal instructions to the other person to build a structure. The other person must rely only on verbal instruction in an attempt to duplicate the structure. Sounds easy, right? It turns out to be rather difficult because when you give instructions, unless they are really precise, the other person's structure will be different. While the terms code and program are generally defined as "a set of instructions," I prefer to define programming (or coding) as constructing an information machine. If the code is on a flat medium such as paper or a computer screen, the machine is created through writing. There are machines, using other paradigms such as data flow, where the machine is drawn rather than written. Others "code" with tangible materials. This idea of generalizing the word "machine" originated with Alan Turing. Turing defined his symbolic artifacts as machine components, with the interacting symbols comprising what computer scientists call a "Turing machine." Coding is akin to making a machine on paper through writing, but more generally, coding is about focusing on the how. How are the colored blocks put together to create a structure? How do you make pancakes? How does your car work? Coding is a type of modeling where the model is something that explains how something works, or how to do something. Today, we code using modern programming languages, most of which are written, but the "code" of the analog Antikythera computer built over two thousand years ago was a bronze gear train.

What is Art?


This is part one of a two-part blog  post on the nature of art and code. We begin with the subject of art. The item above is a bit abstract and does not represent anything in particular, and yet I enjoy the juxtaposed colors, shapes, and material textures. I observed it from different angles, and stared at it for five minutes.  The image is a photograph of a work canvas laying below a brass table in our dining area.  The table was recently cleaned and polished, and I am ashamed to admit that I did not do the cleaning or polishing, but enjoyed the experience nonetheless. And, therefore, we can now surface a key component of art, which is to pay attention, to observe, and to enjoy the experience. John Dewey thought enough on this matter to write "Art as Experience" in 1934. I have always viewed art in this way: as an experience rather than solely as an object. Although, clearly objects of significant cultural value reside in museums, and if I am in any city, I make immediate plans to visit the museums--they are treasure troves of experiences. Time machines to different places and worlds. When an artist creates something for display or exhibition, the artist is saying "I made this experience." Ansel Adams, with his remarkable photography of Yosemite is reaching out to you across time and space, hoping to allow you to have the same experience he had while exploring the wonders of nature. Let's let X equal a work of art that you could find in a gallery, someone's apartment, or in a museum. The questions "who could have [easily] re-created X?", "how famous is X?", "how much is X worth?" are unsatisfactory. If anything, the hope is that we can pay attention, observe, and enjoy 24/7 regardless of where we are. For me, that is real message of art--personal experience, while continually fine-tuning our observational facilities. So, is the above picture a work of art? Yes. I made it for myself and for you. Go out and experience something else.

UX of Mathematics



The acronym "UX" stands for User Experience. What does this have to do with mathematics, and the disciplines that grew out of math (e.g., science and engineering)? What is mathematics? Mathematics is an ancient discipline that formalizes our abstract concepts about the world. So, for example, algebra or set theory are collections of abstract ideas whose realizations occur everywhere. Mathematics has nothing to do with notation. So when you see symbols on the screen, that is not mathematics -- it reflects a standardized, and economical, representation of mathematical thought. These notations have changed over time and are well-documented by Florian Cajori. So, y = ax^2 +b is not math? No, it isn't. This is a modern and efficient way to communicate about math with other people. So, if this expression is not mathematics, where is the math? Mathematics is a mental phenomenon. But having said this, there are now two paths we can follow when we do create representations of mathematical structure: (1) create representations that use traditional notation, or (2) create other representations. Both of these paths are useful, but I would like to suggest that mathematics as humans practice it has embodied interaction at the core. So, if you think about adding two numbers together, there are hundreds of ways to express this, interactively--using your bodily affordances. I mention some of this in section 26.2.2 of a chapter on aesthetic computing. The way in which you express addition is part of doing mathematics. To find a philosophical treatment consistent with this view, read Lakoff and Nunez's Where Mathematics Comes From. Similar arguments can be made for representations in computer science. How does one represent the concept of iteration? It is easy, but problematic, to get caught up in specific typographic notations such as "for (i=0; i<10;i++) {...}" to iterate from 0 to 9, but this has nothing fundamentally to do with iteration. How we express iteration can take on a huge number of forms and varieties. Iteration is a mental concept, which can be realized through multiple, creatively-derived representations. UX lives at the center of mathematics as we practice it. We have a tendency to dismiss the body, at large, in mathematics and computer science -- as if mathematics does not require a body; a disembodied "brain in a vat." And yet, we do have bodies, and a deep understanding of mathematics requires them.

The Beauty of Small Data


I started playing chess at a young age when my uncle in England sent me a tiny plastic chess set for Christmas. What were these strange pieces? How did they move? Before long, I learned that they could make interesting patterns on the checkered board. I followed Fischer vs. Spassky with an almost religious fervor. Over time, I became interested in computer science and followed those who made chess machines and software. And then came the inevitable day when the machine beat the reigning world champion (Kasparov).  What were we to do now? I guess there goes chess out the window. But no. Humans continued to play chess, and the game is as popular, or more, than ever. There a lesson here. Just because we teach machines to excel at artificial intelligence and at machine learning doesn't mean we stop our quest for life-long learning and enjoyment. Big data is hot. The machine can run through an array of sophisticated algorithms so that, for instance, your search engine experience is more meaningful. I am grateful for this capability and the research that goes into it. Think of the massively complex data networks and automated inferences and patterns generated from them. And yet, I find myself interested in teaching students to draw small networks for things that they see around them. By doing this, students learn something about semantic networks and concept maps (ideas developed by artificial intelligence researchers in the 1970s). The learning that occurs is personal and in this case, does not require the big. It requires an attention to detail and a never-ending fascination with discovery.

When Tool Becomes Art


We were in Prague just outside of the castle on the top of the hill. Of course, castles are always on the tops of hills. There was a narrow path leading somewhere, and it wasn't all that clear where. Martha and I decided to make an expedition just to see what was down there. It was a blacksmith's shop. Smoke billowing out of the chimney. On the outdoor wooden table, we could see a variety of jet black objects--but my favorite was a wine corkscrew that doubled as a beer bottle opener. A fellow came out and said that he was closed but would reopen the following day. So, we returned the next day about the same time that a father and his young daughter arrived. They had flown in from California. There were four us. And the blacksmith. He banged on his anvil and was a bit upset that he was not getting something quite right, so back to the air bellows which pumped air into the fireplace and increased the flame. Then, the blacksmith did something I had not expected. He pointed to me and to the young girl and said "Now, you try." We put on gloves, aprons, and safety goggles and went at it. By "at it," I mean simple things such as making something square from a round rod. Then we were able to make a helical, screw pattern from the metal. You twist and twist while the metal is red hot. When all was said and done, I realized that my outlook on the blacksmith and his operation had changed. This wasn't about objects and tools. It was about craft and process. I can still remember the process and how the hot metal felt as I was twisting it. The difference between art and tool is a matter of perspective. A tool becomes an art if you pay attention to it--taking the time to dive into it..to broaden your understanding. I used to think of the blacksmith and his products as tools for my consumption. Some folks think about "technology" like this--a tool for some other purpose. Put on an apron with me and get ready to twist and shape some metal.

Modeling & Coding for Understanding


This is the start of my life in programming. 10th Grade. Downingtown High School in Pennsylvania. My father told me of an after-school class being taught by two teachers from a local college-- West Chester State College as it was known at the time (1970). In my first class, I was provided with a blue/green IBM plastic programming template. I think the first assignment was to write a program that read a count (N) of numbers, averaged these numbers, and then output the average of the numbers. We had a card punch machine in the school, and our card decks were shipped to the College for processing. Later, we received our green-striped paper output. It was a long process. Although some may see programming as a sort of automation, I was creating a new "machine" with each program. Imagine being told that you could create a machine, but that you'd need to "write" a machine rather than build one. This is the essence of programming. Even today, a program is a mental model of an ethereal machine--stuff that moves around, is stored, and manipulated. Writing doesn't really do the "machine" justice. It is really all visual, a mental model, and I used writing because that is all that was available. Today, coders and modelers do the same thing -- they create imaginary machines that are not quite real--all existing in the head, but real enough to reason about unless too many weeks have passed, and the formerly crisp memory of the machine begins to decay. The goal of programming and modeling is not to automate a task but to understand one. What better way of understanding something than to design a machine to recreate that thing? Want to know how to bake an apple pie? Write a program. Create an apple pie machine. As the old saying goes, if you cannot write a program (or make a machine) then you don't understand the task sufficiently.

Computing in the Wild


What is computing? What is computer science? These questions would seem to have easy answers, but the field of computer science is still in its infancy compared with fields such as physics, and the mothership--mathematics. The term "computer science," somewhat unfortunately, seems inextricably linked to a family of artificial devices we call computers. But can you think of any other major discipline with this characteristic? We do not refer to astronomy as telescope science, chemistry as mass-spectrometry science, or mathematics as compass-and-rule science. In mathematics, for instance, educators will observe mathematics at play in nature. In the above image, the mathematical symmetry of the pagoda and the geometric branching structure of trees and bushes are all too evident. And yet, in computer science, we seem fixated on the box. However, computing, with its focus on information, has potentially a much larger role to play in our world. Recently, we created another video that dives into computing to illustrate three major paradigms, ways of seeing information management and flow outside of the box. None of this is to downplay the relevance of post 1940s growth in what we know as computer science today. Computers assist us with our daily chores, our workplace tasks, and our entertainment options. But if computer science is to approach the ubiquity of mathematics in our world, we must venture beyond "code" and back to the idea that computing can be just as much about describing what we see, and how we see it--in the wild through an information lens.

I am a Modelist


If you read this blog, you probably have an interest in automata or modeling, computing representations, or all three. I'd like to spend some time talking about being a modeler. A modeler is someone who models. Modeling can be anything from Danylo Stanchak's model of Elsa to creating models of behavior, process, and shape. The mathematicians also have their version: model theory which falls under the study of logic. Like a linguist is someone who enjoys languages of all types, perhaps a modelist is the modeler-equivalent of the linguist? A modelist is someone who studies modeling independent of a specific focus area or discipline. Perhaps, I am leaning toward the view espoused by G. H. Hardy in A Mathematician's Apology. It is not that disciplines are uninteresting, because they most certainly are essential to highlight the effects of modeling, but modeling itself is even more interesting for me. The disciplines are proof of the concept of modeling. But, unlike Hardy's mathematics, modeling is not yet considered a formal discipline with academic areas and departments (note: there are a few exceptions to this rule). However, there are conferences, journals, and societies devoted to modeling. I am certainly a modelist. Are you?

Modeling Art History


For my modeling & simulation class this Fall, I am exploring a new collaboration with colleagues from the Dallas Museum of Art (DMA).  The exploration consists of a mix of digital humanities, art history, computer science, and modeling & simulation. The above image is of a tunic that is part of the DMA collection and currently on display in the exhibition entitled Inca: Conquests of the Andes/Los Incas y las conquistas de los Andes. It is a beautiful piece with rich history from the Peruvian culture. What does this have to do with process modeling that we do in simulation? The tunic, like all works of art, can be interpreted and presented in numerous ways. I think this diversity of interpretation is central to both modeling and to the humanities. Bruno Latour, in How to Be Iconophilic in Art, Science, and Religion [1], concludes his essay with "The difficulty is to learn how to be iconophilic for one form of visual culture without being iconoclastic for the others." We need diversity in interpretation, and modeling helps achieve this diversity with multiple, mediated objects each providing an understanding of a phenomenon, like the checkered Andean Tunic. How might we model a process that partially re-creates this pattern? This can be done with text-based scripts or with a visual program. How did the Peruvians weave this particular tunic (ref. Lesli Robertson's gallery talk)? One classic technology employed is the back-strap loom. How might we use modern weaving methods and weaving draft notations to model something similar? How were the red and black dyes made for the pattern? What transportation processes (people and things moving around) were in place to get materials to and from their locations? All of these are questions of process [2], and models of process are therefore creative interpretations of the tunic. Process models can be an integral part of the history and interpretation of art. In last year's class, we used Max/Msp (a visual programing data flow program used by artists) and the target scenarios to be modeled came from everywhere. This year our targets are all inside the DMA. We are therefore continuing to use Max, but diving further into the art world for our cultural context. Can we learn to see these models in the art? Can diagrammatic models provide additional interpretation and knowledge about art and culture? This approach is a departure from "Big X" (e.g. Big Data).  We may find some interesting workflow models that yield new information on art by amassing and sifting through huge online collections. This should be part of our process. But, to quote Feynman in the spirit of "close reading" within the traditional humanities, there is plenty of room at the bottom if we only diversify our interpretations of a single work.

[1] In Latour, B. 1998. How to Be Iconophilic in Art, Science, and Religion? in Jones, C. A. & Galison, P., Eds, Picturing Science Producing Art, Routledge.

[2] Questions of process are not only central to modeling and simulation but also to computer science. The term "code" means to model a process, usually with typographic symbols in the form of a program or script.

Exploring Computing Representation

%d bloggers like this: