Category Archives: computing

You Don't Know Jack

Love Jack? Turns out to be an icon at the university where I work: University of Texas at Dallas. To be perfectly honest, none of us really knew Jack before the start of the Spring 2016 Creative Automata class, however, because of my students' admirable diversity, we can really know more about Jack. This is the biggest project to date, and the Jack adventure began with a challenge to the students to explore the sculpture (which is located in the courtyard of the ATEC building) from different perspectives. What perspectives could be better than those that are natural to the students' interests and backgrounds?  The class contains computer scientists, artists, and designers--often with students crossing over among areas and interests. A smartphone app accompanies the main Jack web page, so that when one is near the Jack, the perspectives can be browsed. Interested in the artist's history, how Jack relates to computer science and mathematics, modeling & simulation,  graphic design, digital fabrication, the connection of art to science? It's all there. A beacon is next to the sculpture to facilitate object-based learning discovery across STEAM (Science, Technology, Engineering, Art, and Mathematics). You can also start with STEAM and browse based on interest.

Why the STEAM Argument is One-Sided


Full steam ahead. Or should I say STEAM ahead? STEM stands for Science, Technology, Engineering, and Mathematics and has been a driving force initiated by the National Science Foundation to focus education policy within technical areas and their associated disciplines. More recently, the letter "A" has been added to create a new movement called STEAM. The "A" stands for the arts, and according to a leading site devoted to STEAM, STEM + Art = STEAM. Since I spend much of my time thinking about the interconnections between STEM and the Arts, I welcome the STEAM movement. And yet, I have deep concerns about the movement's three published policy goals stated on the STEAM site:  (1) transform research policy to place Art + Design at the center of STEM; (2)  encourage integration of Art + Design in K–20 education; and (3) influence employers to hire artists and designers to drive innovation.  These are worthwhile goals, but notice how all three goals seem to be about getting STEM-oriented folks to hire artists and designers, and placing art & design at the middle of STEM? Let's flip this. What about having STEM at the center of Art and Design? I am not suggesting doing away with the three STEAM goals, but I am recommending some sort of balance by extending or broadening these goals; the current ones are lopsided. I strongly advocate new ways of starting with design and the arts, and then surfacing STEM concepts from within art and design. For the STEM subset of computing, this advocacy resulted in the aesthetic computing movement. Recently, this approach has taken root in learning systems thinking in the art museum.  I am not the first to suggest this if we consider the larger literature base of  blending STEM with the Arts. Take Martin Kemp's book The Science of Art where he explores mathematics and optics via art. Also, the MIT Press Leonardo journals edited by Roger Malina has extensive historical coverage of intersections of STEM and the arts.  Leonardo was founded in 1968, and so its publications contain a treasure trove of knowledge, suggesting new ways to get to the heart of STEAM.  To advocates of STEAM, my suggestion is to rethink of STEAM as two-way traffic: two steam locomotives, two tracks, perhaps with some switches here and there.

Computing in the Wild


What is computing? What is computer science? These questions would seem to have easy answers, but the field of computer science is still in its infancy compared with fields such as physics, and the mothership--mathematics. The term "computer science," somewhat unfortunately, seems inextricably linked to a family of artificial devices we call computers. But can you think of any other major discipline with this characteristic? We do not refer to astronomy as telescope science, chemistry as mass-spectrometry science, or mathematics as compass-and-rule science. In mathematics, for instance, educators will observe mathematics at play in nature. In the above image, the mathematical symmetry of the pagoda and the geometric branching structure of trees and bushes are all too evident. And yet, in computer science, we seem fixated on the box. However, computing, with its focus on information, has potentially a much larger role to play in our world. Recently, we created another video that dives into computing to illustrate three major paradigms, ways of seeing information management and flow outside of the box. None of this is to downplay the relevance of post 1940s growth in what we know as computer science today. Computers assist us with our daily chores, our workplace tasks, and our entertainment options. But if computer science is to approach the ubiquity of mathematics in our world, we must venture beyond "code" and back to the idea that computing can be just as much about describing what we see, and how we see it--in the wild through an information lens.

Formalized Synesthesia


I recently engaged in a three-way podcast conversation covering research that we do in the CA lab, as well as activities in the Creative Automata class that I teach--if that is even the right word. Guide? The title of this post is gleaned from Christopher White who works with Elecia White. I engaged in dialogue with both of them, and thoroughly enjoyed our discussion. Elecia and Chris produce a podcast called Embedded where the main theme is embedded systems and electronics. But they tackle a wide variety of interesting topics around this central theme. This audio podcast name Bubblesort Yourself was invented by Elecia, and the hour long podcast can be found here. Their Embedded podcast can also be accessed using the Apple podcast app or the equivalent app on Android phones and tablets. I listen to their podcasts regularly, and also to other podcasts while I take long walks. For some of you, driving the car or working out in the gym may be good times for podcast listening. Chris White also posted an accompanying blog entry where he expands upon formalized synesthesia. Is that what we do when we model in simulation? It seems to be on the basis that we employ many models, each of which contains a hidden set of analogies. The models are encoded with respect to our senses [credit: artwork Synesthesia above is from Nuno de Matox].

Design It For Yourself


Design is a big word, and something we all feel passionate  about. We know from Jonathan Ive of Apple that well-designed things can enrich our lives and, indeed, do quite well in the marketplace. Think of products such as the iPhone, iPad, and the upcoming iWatch. These products are well designed by Apple, and meant for you, the consumer. There are ways to customize the look and feel of the human interface in these devices. But, is it possible for people to design things for themselves? Yes, but for a different type of market: self-education. Imagine that you are in a class, trying to learn something hard like computer science or calculus. Further imagine that the teacher, rather than dishing out pre-designed computing and mathematical structures plays the role of facilitator, allowing you to design your own objects. Design your own code. What would it look and sound like? Design your own integrator. Make your own personal language. Design your own representation for equations. This isn't about markets and sales. It is about allowing you to craft your own self-inspired representations--as a way to promote self-interest and creativity--you may come to learn better because you have been given an opportunity to create rather than to interpret the symbols of others. This approach of designing something yourself to learn something goes by another name: art. Let's promote learning by creative representation and creative design. Design, in this particular instance, not of creating something for other people, but creating something because it moves you.

Personalized Model Learning

Property release:3 Model release:Q,E,J


Much of what we do in the Creative Automata (CA) Lab is oriented around multiple representations of a single abstract mathematical concept--such as integration in calculus or sorting in computer science. How can we personalize approaches for learning something like integration? Is it possible to leverage our multiple cultures to engage and motivate the learner? The lab just submitted our video entry to the National Academy of Engineering (NAE) Grand Challenges for Engineering Video Contest called E4U2. Sharon Hewitt from the CA Lab designed and produced this video. The video segments include representations of a virtual analog computer based on the sand-like flow in PowderToy, as well as several personalized models of the Lotka Volterra model. Instead of  making models for other people, consider that you can learn about modeling by making these wonders for yourself. In this arts-based approach, you will also interest other people in modeling.


Big Process


Big Data: it is (still) all the rage. And rightfully so because we have access to more data than ever before. The "bigness" of data refers mainly to the sheer magnitude or volume of data available for our consumption. This volume has increased exponentially and shows no sign of abatement. But data indicates one side of the coin. Process is the flip side. Without process, the data repository is a large pile of undifferentiated pieces. Even a "data structure" is a process in disguise since the structure reflects a procedure that must read and write to this structure--thus creating it. This idea of process, in a miniature scale, can be seen in programming. Once upon a time, the program and the data occupied the same memory space, and to the unaided eye, the computation looked like a collection of hexadecimal bytes -- which ones were the data and which were the operations? Who knew? A careful deciphering and knowledge of the opcodes showed us the way. Today, the idea of "process" is essential if one has lots of data. How are the data sensed, processed, merged, diverted, massaged, and transformed? Like a petrochemical plant, the raw materials (e.g., data) are useless without a clearly engineered, and formally represented, process flow. Big data needs big process and big model.

Being an Understander


You have probably heard the old canard, "Those who do, do. Those who cannot do, teach." Time to set the record straight. Pick a topic that you think you know. Any topic. Now, explain it to 3 different people: a child, a colleague not in your area of expertise, and someone who is completely outside of your social circle--perhaps someone of a very different age than  your own. If you cannot explain your topic to them, then you don't know what you are talking about! I know that this may come as a bit of a shock to some--that learning and communicating are flip sides of the same coin when it comes to mastering knowledge. But the classic Greek philosophers had no problem with this. Plato (a student of Socrates, image above) and Aristotle got it. They invented it. Plato and Aristotle formed what would 1600 years later would be called colleges, schools, and universities. The Academy was created by Plato around 387BC. Aristotle, who studied there, formed his own: the Lyceum. Luminaries such as Einstein and Feynman also got it. Look at how they behaved, how they communicated, and what they accomplished. So, what is the lesson here? Stop trying to impress only your colleagues and start communicating what you know to the other 99% of the human race. By doing so, you will dramatically take your own understanding of an area to new heights. You will not only be a Teacher, but an Understander.

We Are All Technologists


I was struck by text that I recently saw when browsing online information about art-science collaborations and upcoming conferences. There was a phrase in one of them promoting connecting "artists with technologists." What the heck is a technologist? Everyone is a technologist. A "technologist" is someone who uses (modern, recent) technology. One might think that a computer scientist or electrical engineer is a technologist, but think again. People in these fields have their own research and agendas; they do not think of themselves as technology gurus out to serve the public at large.  They use technology like everyone else. I have to backtrack a little bit because it so happens that these fields just so happen to produce the most sophisticated technologies (as end products) around. But to a computer scientist, a digital artist might be a "media technologist." The message here is: let's penetrate the broad term of "technology" and get to what is really important to individual practitioners of art, computer science, and engineering. Technology, and the "digital", are simply tools for all of us to do that which drives our passions.

Performing Abstract Concepts


I was in Vienna about a week ago at the SIMULTECH 2014 Conference, and gave one of three keynote talks. I spoke about why computing is everywhere, and that when we teach it, or think about it, we need to re-emphasize analog, in addition to digital, computing methods. It is the analog that enables us to link to the real world. In a prior post, I covered how to portray one computing concept (queuing) within a media-rich environment. I used this example in Vienna. The result was a sort of performance of the abstract queuing object since musical instruments were being queued. This makes me wonder about whether we should perform other mathematical or computing constructs? The idea is the reverse, the complementary case, where artists use computing as a means to create music and art. In this instance, it is the abstract concept of queuing which is placed in the foreground--that which is to be experienced and appreciated. We can present lots of material in this way. Bubblesort performed by Hungarian folk dancers is a great example.

Sonifying Modeling and Coding

Screenshot 2014-08-20 17.07.55

One of the first ways to learn computing is through coding using a programming language. Educators tend to favor one language over another, and it may change from year to year. At UTD, we are starting off non-majors with Javascript this year since Javascript has a good facility with human interaction. The example above is from a library called JSXGraph. A serious challenge for all introductory courses in computing, even when using Javascript, is sound. Many packages now have sound libraries but these libraries are often quite primitive in terms of what you can do. This situation may be changing. Charlie Roberts of UCSB has created a Javascript library for his fairly high level Gibber. Gibber was originally designed as a "live coding" environment, but now that there is a library, it means that we can write Javascript anywhere and include more complex sounds--allowing us to hear mathematics, computing, and scientific concepts. Here is something simple I worked on that connects a simple example from JSXGraph and Gibber.  This Gibber code uses granular synthesis. You can also just click on the above image to take you there.

Computing as Science


My last post referred to the TEDx talk given a few weeks ago at the University of Texas at Dallas. The subject was "Computing Everywhere." This talk was based on a paper I wrote for the 2014 ACM SIGSIM-PADS conference recently held in Denver. The paper entitled "Computing as Model-Based Empirical Science" can now be found in the ACM Digital Library. To see computing everywhere, or to see it as a science like physics or biology, the first step is to distinguish "computing" from "computer." Computing is based on information structures and processes. Computers use whatever technologies are available (since the 1940s, electricity and electronics) to facilitate computing. The argument for the Universe as a Computer, under the name Digital Physics, is a related hypothesis although at a finer level of granularity.