Category Archives: music

You Don't Know Jack

Love Jack? Turns out to be an icon at the university where I work: University of Texas at Dallas. To be perfectly honest, none of us really knew Jack before the start of the Spring 2016 Creative Automata class, however, because of my students' admirable diversity, we can really know more about Jack. This is the biggest project to date, and the Jack adventure began with a challenge to the students to explore the sculpture (which is located in the courtyard of the ATEC building) from different perspectives. What perspectives could be better than those that are natural to the students' interests and backgrounds?  The class contains computer scientists, artists, and designers--often with students crossing over among areas and interests. A smartphone app accompanies the main Jack web page, so that when one is near the Jack, the perspectives can be browsed. Interested in the artist's history, how Jack relates to computer science and mathematics, modeling & simulation,  graphic design, digital fabrication, the connection of art to science? It's all there. A beacon is next to the sculpture to facilitate object-based learning discovery across STEAM (Science, Technology, Engineering, Art, and Mathematics). You can also start with STEAM and browse based on interest.

Formalized Synesthesia


I recently engaged in a three-way podcast conversation covering research that we do in the CA lab, as well as activities in the Creative Automata class that I teach--if that is even the right word. Guide? The title of this post is gleaned from Christopher White who works with Elecia White. I engaged in dialogue with both of them, and thoroughly enjoyed our discussion. Elecia and Chris produce a podcast called Embedded where the main theme is embedded systems and electronics. But they tackle a wide variety of interesting topics around this central theme. This audio podcast name Bubblesort Yourself was invented by Elecia, and the hour long podcast can be found here. Their Embedded podcast can also be accessed using the Apple podcast app or the equivalent app on Android phones and tablets. I listen to their podcasts regularly, and also to other podcasts while I take long walks. For some of you, driving the car or working out in the gym may be good times for podcast listening. Chris White also posted an accompanying blog entry where he expands upon formalized synesthesia. Is that what we do when we model in simulation? It seems to be on the basis that we employ many models, each of which contains a hidden set of analogies. The models are encoded with respect to our senses [credit: artwork Synesthesia above is from Nuno de Matox].

Performing Abstract Concepts


I was in Vienna about a week ago at the SIMULTECH 2014 Conference, and gave one of three keynote talks. I spoke about why computing is everywhere, and that when we teach it, or think about it, we need to re-emphasize analog, in addition to digital, computing methods. It is the analog that enables us to link to the real world. In a prior post, I covered how to portray one computing concept (queuing) within a media-rich environment. I used this example in Vienna. The result was a sort of performance of the abstract queuing object since musical instruments were being queued. This makes me wonder about whether we should perform other mathematical or computing constructs? The idea is the reverse, the complementary case, where artists use computing as a means to create music and art. In this instance, it is the abstract concept of queuing which is placed in the foreground--that which is to be experienced and appreciated. We can present lots of material in this way. Bubblesort performed by Hungarian folk dancers is a great example.

Sonifying Modeling and Coding

Screenshot 2014-08-20 17.07.55

One of the first ways to learn computing is through coding using a programming language. Educators tend to favor one language over another, and it may change from year to year. At UTD, we are starting off non-majors with Javascript this year since Javascript has a good facility with human interaction. The example above is from a library called JSXGraph. A serious challenge for all introductory courses in computing, even when using Javascript, is sound. Many packages now have sound libraries but these libraries are often quite primitive in terms of what you can do. This situation may be changing. Charlie Roberts of UCSB has created a Javascript library for his fairly high level Gibber. Gibber was originally designed as a "live coding" environment, but now that there is a library, it means that we can write Javascript anywhere and include more complex sounds--allowing us to hear mathematics, computing, and scientific concepts. Here is something simple I worked on that connects a simple example from JSXGraph and Gibber.  This Gibber code uses granular synthesis. You can also just click on the above image to take you there.

Sensing Simulation Models


When I teach modeling and simulation, I tend to focus mostly on the structure of models. I start with a thorough discussion of time and systems concepts, and then move on to cover different sorts of dynamic models in both discrete and continuous space. In the past, I had relied on using examples purely from the real world in emphasizing the importance of modeling. For example, all fast food restaurants, manufacturing lines, and theme parks have one thing in common: queuing networks. A queuing network is a dynamic model abstraction of what happens in these things: objects (often people) wait in line, get served, and move on.  In teaching queuing, and other, models, I am trying something new this Fall. I am starting with human-interaction and media being the means by which to get students interested in modeling. For example, the single server queue (SSQ) shown above has an operation that can be both seen and heard. This one is programmed in Max/Msp (which is a visual language with strong roots in music, imagery, and video). The SSQ is experienced, by tying events to an audio synthesizer. Everything is in software. An SSQ becomes an audiovisual instrument.


Machines of the Heart


I took this picture in my kitchen a few weeks ago when it was frigid outside and two hot chocolates seemed a necessity for survival. The photograph shows two coffee cups on a granite counter top. You can see the reflection of the lights off of the granite. The light patterns (i.e., caustics) that are visible inside of each cup are a result of the light interacting with the inside of the cups. Each cup has two cardioids because there are two overhead lights.  Models that involve ray tracing, or approximating light with lines, produce some really interesting patterns. The collection of two light sources and cups forms an analog computer, essentially, for gaining insight on these and related "envelopes" of multiple lines crossing. The sum of places where these crossings, or intersections, occur forms the cardioid patterns. For those readers interested in sound, some microphones use cardioid patterns as a means for defining what angles are picked up by the mic.

From an automata, or machine, perspective the cups form a model of the mathematical cardioid concept, with the system input being light, and the output being the pattern you see with your eyes. You may object to my use of model here by my mentioning that the cups are models of a mathematical concept, and you would be right to object since this philosophy is a bit controversial; a subject of one of the upcoming posts. Thinking of mathematics as the ultimate target places the activity of modeling at the core of cognition.


Music Box Thing


What is it? Looks like a hair comb and a strange cylinder with small bumps. This is a 3D printed music box. I've been thinking quite a lot lately about 3D printing ever since I opened up my lab to show posters and research last Thursday, and then realized that everyone was, instead, taking a bee-line to our 2 Makerbot 3D printers. "Is this the room where the 3d printers are?" Sure! Why not? It seems that there is a new 3D printer company on KickStarter every day. So, clearly, this represents a technology that is only going to get better and cheaper. Lots of things can be fabricated with a 3D printer, including the music box. Our interest is a bit beyond music, though, to the point where we are looking at objects like the music box and thinking about how we can use these quick-print objects for math and computing education. There are many possibilities -- people are drawn towards the tangible, the touchable, and that which can be forged immediately from plastic. I am reminded of a toy when I was young, where we had a kit for making plastic insects. What ever happened to it?


Organum Hydraulicum


Athanasius Kircher was a 17th century German jesuit scholar who produced many publications on machinery. One of them, Musurgia Universalis, was written in 1650 and described ways of creating automatic music. The machine shown above, a water organ, is from Book 9 of that work. Water and air are introduced at the right side into a vessel termed a camera aeolis. The rate of water flow introduces a displacement of air which exits into the vertical pipes comprising the musical organ. The control of which notes are produced, and when, is achieved through barrel rotation.  The water not only displaces air for the organ, but also drives the barrel. The barrel has protrusions that interact with the keys as the levers interact with the protrusions. There is a separate mechanism shown in the upper left of the illustration.