Archive for category Physical Computing

3D Scanning and Re-Presentation Final Projects

In the past few weeks, I have taken a whirlwind tour through the worlds of 3D scanning & position/orientation sensing. The product of this study is still a work in progress.

Here is my project concept statement:

If Photorealism is only one of many possible styles of painting, then the “realism” of our natural vision must similarly be one of many possible modes of realism. There is no reason to suspect that our natural vision presents the world as it really is. We are abstractly aware of this and yet are limited in our ability to experience other modes. It is the goal of this project to enable the user to experience a physical environment in a visual style which differs from his or her natural mode of vision. This is achieved through the use of a 3D scanner, which images a physical space and provides a software system with a cloud of three-dimensional coordinates. The software uses geometric analysis to interpret these points into an alternate view of the space – a view which may be described as a blocky, Lego-like version of the physical environment. This alternate view is then sent to the user’s head-mounted display. A 3D positioning and orientation sensor is used to locate the user within the depicted space. This system is intended for use in an interactive cinema experience, entitled “Parallax Digitalis,” which will use prerecorded and interactive sequences to convey the story of a young man entrapped by the false notion that he is a deity. The current implementation of this project serves as a proof-of-concept for my longtime theoretical exploration of the possible uses of descriptive geometry in the telling of cinematic narratives.

Here are the results so far:

Bust (Original Data) Card Detection

Bust Stage Decomposition

Building Blocks

And the steps taken to reach these results… (continue on next page)

Read the rest of this entry »


Week 7 Lab – Multiple Serial Outputs

Here is a simple double joystick game controller similar to that used by Playstation.

Double Joystick / Arduino Serial data format (Arduino sketch)

Double Joystick / Processing data reading (Processing sketch)

Game controller

Game controller (detail)

For the media controller group project, I wrote the following code, which uses a potentiometer and a force-sensing resistor to recreate a turret gun in the style of a classic arcade game:

Potentiometer/FSR Serial Data Formatting (Arduino sketch)

Potentiometer/FSR Turret (Processing sketch)

I am now working on a hot-swappable version, which allows up to 6 players to plug their controller into the game at any time. The turrets will be interfaced with a game and generative audio environment, which are currently being developed by my teammates Nien and Don. Having never worked for a large software company, I’ve been somewhat mystified by the logistics of team development. This group project has been a great learning experience for me. I’m very pleased with our collaborative efforts.

No Comments

Check(-out) yourself

An increasing number of supermarkets and stores such as Home Depot are making use of self-checkout aisles. The very presence of these devices seems to imply an offer of increased efficiency to the shopper. Surely this is an innovation and innovations tend to offer some form of positive change. Yet, we must ask what aspects of the ordinary check-out procedure in particular will be bettered by the absence of a clerk. What inefficiencies are inherent to this sort of human interaction? The conversation? The barcode scanning? The exchange and counting of cash? In days of yore, arithmetic might have slowed such financial exchanges. But this difficulty was overcome long ago with the change-calculating cash register. Everything else is as inefficient at a self-checkout as it is at an ordinary one. Except that the self-checkout requires fewer employees and this, of course, is the point. There is much that could be said about the social and political ramifications of this innovation, but for this post I will stick to the technological implications. What does it mean for a store to push check-out responsibilities onto the consumer? Foremost, it requires a technical interface that requires no training. But does such a thing exist? Look at the blinking VCR display at your grandparents’ house. I dare say that the iPod interface is seen as universally comprehensible only because those who would have trouble with it were already scared off by the term “mp3.” In the case of self-checkout, we may assume that a large number of people who would have trouble with the interface will make no attempt to use it and will instead rely upon the traditional form of check-out. We may also assume that ambitions will run high amongst another group that is not really up to the task and that these people will reduce the overall efficiency of self-checkout, perhaps so much so as to make it a fruitless enterprise for everyone. In any case, we should be relieved to know that at least one clerk will remain on the workforce as a monitor and facilitator of self-checkout operations.

I went to Home Depot, one of the most poorly-run establishments in the world, to see how self-checkout was helping or hurting their business. Home Depot seems to be the perfect place for self-checkout because in my experience the vast majority of the store’s employees know no more about the inventory – what it is, where it is, if it is, etc – than the customer. Before observing other customers using the self-checkout, I thought I’d give it a shot myself. I scanned several items with no trouble. But disaster struck when I attempted to scan a small plank of wood. The database was unaware of the product. The attendant came over and tried to scan the plank’s barcode. Nothing. So he called the lumber department a few times. Eventually an elderly man came over from the lumber department with a handwritten list of prices that spanned several crumpled pages. Yikes, I thought. But I was wrong, this man found the price of my plank almost instantly on his crumpled pages. That taught me a lesson.

My initial priority in observing the self-checkout was to ascertain what sort of customer chooses this method over the traditional one. The most significant factor affecting this decision seemed to be the number of items in the customer’s cart. The majority of people in the self-checkout aisle had only a few items, which seemed to reflect a belief that scanning a few items yourself would be quicker than contending with the much longer lines in the human-checkout aisles. More than any bias of age or technical knowhow, there was a noticeable bias of demeanor – the customers who chose the self-checkout appeared to be in a more of a rush than the others. The self-checkout was empowering a “if you want something done right (or quickly), do it yourself” attitude. Whether this belief was justified is hard to say. There were any number of hold-ups at the self-checkout. The two most common problems were the initial confusion of where to begin and confusion over the machine’s stipulation that an item must be placed on the bagging platform after being scanned. Interestingly, by the second or even third scanned item, a number of customers were still neglecting to abide by the proper bagging platform procedure despite repeated scolding by the machine. The majority of customers paid with credit cards, but for both credit and cash payments, the financial transaction tended to be the smoothest step in the process. This was not surprising since it may be assumed that the average consumer interacts with pay stations of all kinds on a highly regular basis. Though their dealings with other elements of this procedure may be substantially more limited. It was surprising, however, to see that most customers were not perturbed by the computer interface in general. Clearly this type of interface is also reaching ubiquity. The greatest challenges were procedural ones – issues of how the software instructed the user to engage with the hardware. The computer voice asked the customer to move a scanned item to the “bagging area” and displayed a legible graphic of this. Yet, there was something very disorienting about this request. Why? Perhaps it was an issue of language – the term “bagging area.” But I think it was something else. I think the problem with this element of the exchange was that the customer did not understand why this step was so explicitly necessary and so they were delayed as they tried to comprehend the purpose of what was being asked of them. Presumably, this is a theft-deterrence measure and so it is easy to imagine why the customer may not immediately grasp the purpose – they don’t suspect that they are under suspicion. In observing this element of self-checkout, I came to see that a smooth interaction with a device requires the user’s comprehension of why something is being asked of them. A user is reluctant to carry out a procedure, particularly a financial one, when they are uncertain of its purpose. An explanation of the what apparently cannot replace an explanation of the why if it is not self-evident.

The length of the average transaction was between three and five minutes, but many of these were one or two item checkouts. One might expect that each additional item in a cart would add an increasingly negligible amount of time to the overall procedure since the overhead of orienting oneself to the system is the most time consuming element. To the contrary, I found that the sense of hesitance or confusion had not fled from the customer by the time they reached the last item in their cart. Each item was its own hazard. For a few customers, the first item or two went smoothly, creating a false sense of security, which was soon replaced with dread when the third or fourth item could not be found in the database or the barcode could not be read. The latter was a common problem as Home Depot does little to maintain the careful storage of merchandise in its stores. On the whole, however, I was surprised to see that self-checkout was running quite smoothly. I think it is impossible to understate the importance of the fact that the self-checkout customer is a self-selecting one. If all customers were forced to contend with the machine, it would be far more disastrous.

No Comments

Color Learning

The relationship between a color value and a name is a learned and subjective one. A person comes to the conventional associations – a general sense of what values are called “red,” for instance – by repeated exposure.

Color Hardware Color Hardware

Hardware Version source code for Processing (requires Standard_Firmata to be loaded on the Arduino)

Since the hardware version requires… the hardware pictured above, I’m also including a software simulation version:

Software Simulator (version 4).


COLORNAME RVAL GVAL BVAL (example: “red 255 0 0”)

guess RVAL GVAL BVAL (example: “guess 255 80 80”)

rename NEWNAME (works on previous guess. example: “rename pink”)

show NAME (example “show red”)

No Comments

Week 3 Lab & Pet Trick

This week, with the newfound glories of Ohm’s Law and so forth, I feel pulled in many directions. I got Code by Charles Petzold from the library and have been toying with the idea of building a 4 or 8 bit computer. On the other hand, I am starting to think about the final project and how I should start working with gyros, accelerometers and LCDs as most of my ideas so far for this project involve one or all of these components. From everything I’ve been reading, it seems that the latter will require a bit of the former in order to work smoothly and so my intention now is to pursue the best of both worlds… the classics and the modern.

As a pet trick, I’ve been working on making a calculator that converts from integers to binary, handles (at least) addition and subtraction, and then converts back to integer form. Converting from integer to binary is a simple procedure. In binary, we have the following columns: [128][64][32][16][8][4][2][1]. We take our integer and start at the left column. If the integer (let’s say 129) is greater than or equal to the value of the first column, we turn that column ON, then subtract the column value from our integer (129-128 = 1) and then move to the next column. Since the next column (64) is greater than our remainder of 1, we keep moving forward. In the case of this example, we keep moving forward until we reach the 1’s column. We turn that column ON and then our remainder is 0, so we’re done.Since we’ve been on such a potentiometer kick, I decided that the input mechanisms for my calculator would be two potentiometers. This is a bit tricky because the analog values coming from a potentiometer are not totally static and its hard to get the exact value you want. I experimented with mean and median-based methods of “smoothing” the analog input. However, it was a mode-based approach that ultimately worked best for me because this tends to give the most stable (or recurrent) value as opposed to the most central value as is given by the median. Even so, a potentiometer is not exactly the ideal input medium for a numeric value. Below is the source code for a single potentiometer integer to binary converter and display. More on this “pet trick” to come…
Integer to Binary Potentiometer Display Processing Sketch

No Comments

Fantasy Device

In the third week of physical computing, working with hardware still feels quite new to me. This is great because even the must fundamental devices – blinking lights, variable resistors and so forth – seem to hold endless possibilities. That is to say, I don’t feel jaded. In software development, I am already capable of building a somewhat complex system, given the time and thought to do so. As a result, the prospect of building something of a Photoshop (or let’s say Kid Pix) imitation is not all that thrilling – I can imagine the sorts of data structures, the UI and so forth that I would need for a project of this kind. But I have not been thinking about the fine points of hardware development for very long and so the prospect of considering preexistent technologies in this field for the purpose of learning about their component parts and the synergy between them feels beneficial. As Bob Dylan astutely noted, “To live outside the law, you must be honest.” I believe that taking a deep look at the fundamental building blocks of hardware – from potentiometers to digital i/o with a microcontroller – and thinking about how these fundamentals combine into more complex devices is the best way for me to come to new ideas in this field. I have much to learn about leveraging hardware components in clever ways. I think of my first introduction to the notion of recursive algorithms. What a clever and inspiring idea! When you apply it to your own premises, you start seeing entirely new applications and methodologies within your own preexistent concepts. I guess all of this is to say that in learning hardware, I don’t want to rush towards the first semblance of a new idea I might have. I want to take the time to explore the grammar of the medium because this will enable me to reach not only new ideas but also my best new ideas.With this in mind, I propose two fantasy devices, which are technologies that already exist. The spin, however, is that I have chosen technologies that I believe can be produced (at least as proof-of-concept devices) for much less money than their commercial counterparts by taking clever approaches to the fundamentals of electronics that are available to us at the introduction of this course. The devices are the virtual camera and the 3D vertex-scanning stylus. My aim is to consider how these devices can be built in some form using the means available to us in Physical Computing. First, the 3D vertex-scanning stylus (which is not its commercial name, or a particularly catchy one for that matter). Modeling objects can be quite difficult in a 3D environment. So, in some cases, it is easier to build a physical model in clay and then scan the geometry into the computer. Nowadays, the film industry uses 3D scanners for this task. But for a long time, another device was used – a stylus that you ran over the surface of the physical model and collected volumetric data in the manner depicted in the drawing below. Ed Catmull (a founder of Pixar and a forefather of 3D graphics) used a device of this kind in his early work in 3D modeling. I believe a crude version of this device could be built with little more than a series of potentiometers, which are used in determining the position of the stylus in relation to the physical model. The device would work in the following manner. A sculptural model would be placed on a Lazy Susan. The bottom of this Lazy Susan would be attached to a potentiometer so that as the model is rotated, we may track its angle of rotation by utilizing the analog value reading from this variable resistor. At some given point along the circumference of the Lazy Susan, an armature is mounted. This armature has 3 “bones” and 2 “joints.” The length of each bone is fixed and known to us and each joint is formed by a potentiometer which enables us to take a reading of the angle between two bones. When the tip of the armature stylus is pressed against a point on the sculptural model, a button is triggered which tells the device to take a reading of each of the three potentiometer values. With the knowledge of these angles and of the length of each bone, we may compute the location of the stylus tip in 3D space and may therefore associate a point on the model with a Cartesian coordinate in a 3D environment. There are some difficulties to machining the joints (anyone know how to spot weld?) and the degree of freedom will depend on the capabilities of available potentiometers, but I believe this device can be built at a relatively low cost.

Vertex Stylus

My other fantasy device is a Virtual Camera. This device would take the shape of a camcorder. But it would have no lens or image sensor. Instead, it would use gyros and accelerometers to collect data on the camera’s movement through physical space. Using a wireless transmitter, this data would be sent back to a computer and would comprise the location of a camera within some virtual scene in a 3D environment. The camera’s perspectival view of that scene would then be sent back to the LCD screen of the Virtual Camera as a video signal. A potentiometer would serve as a focus ring and a set of buttons would enable virtualized implementations of other real world camera features. The virtual camera can be moved in physical space but correspond to a virtual space depicted on its screen, allowing filmmakers to apply a natural, physical camera style to CG or special effects films. The device can also be used in a Pre-Viz stage of production for the purpose of allowing a director or cinematographer to experiment with and develop a particular shot style. Peter Jackson used a device of this kind for Lord of the Rings. Any commercial implementation of this tool is extremely expensive. Yet, the necessary components are quite available to us and the main challenge in producing such a device would be to fine tune the algorithms used in interpreting the gyro/accelerometer data. I would very much like to build both of these devices this semester. Though, each could be a big project and I want to leave myself open to the prospect of building a final project that does not already exist in any form.


1 Comment

oPP Nano Just Announced!

Introducing our thinnest Optical Polarization Potentiometer ever.

The Nano

The primary components of this model are a nail, a metrocard, a polarization filter dial, photocell and a second polarizing filter (not pictured).


A sample graph:

slimLine Optical Potentiometer Graph

No Comments

Week 2 Lab Project

For this assignment, I started with the idea of building a crude “CD player” with a rotating cardboard disc, 3 pen lasers, 3 photocells and a modified version of the multiplexer I built for last week.Building from Tom Igoe’s Graph code, I produced the following visualization of a single photocell test:

CD Sensor Graph (proto)

I wrote an ambient light normalization procedure in order to maximize the on/off effect (represented here by green/red respectively), which will act as a trigger for the software demultiplexer. The procedure is very simplistic and needs work. As of now, it uses the first 24 readings to find an average ambient level. Any subsequent reading which is greater than 1.1 times the average ambient level is triggers the on state. I would like to maximize the viewer’s ability to see the machinations and so casing the device in a black box is not an option. Here are the possible sensor states depicted on a sample “CD.”



I’ve decided not to pursue the CD player for this assignment because there is too much overlap with my previous project and I’d like to explore some other possibilities. As a foundational variable resistor exercise, I built the following resistor array:


which yielded this graph:

Resistor Array Graph

Naturally, a resistor array doesn’t give you a fluid resistance transition like a potentiometer. But it provides a series of discrete values – 10K, 4.7k, 3.3k, 1k, 470 ohms, in the case of my demo model – to a single analog input on the Arduino and can therefore be used as one axis of a touchscreen-like device. The demo model requires a charged stylus. I’m looking into resistive and capacitive approaches to actual touchscreens, but so far my aluminum foil, wax paper and plastic wrap tests have yielded no results. I also tried using a forcefully drawn graphite line on paper as a potentiometer-style variable resistor. This worked, but the resistance was extremely high even with an extremely short line and so it was difficult to gather usable results. I then tried a mechanical pencil lead, which was somewhat better but had too little resistance. A precise mixture is clearly the defining feature of a resistor. I also did some experiments with a flexible sensor.But something was missing from my project… So far, with possible exception of a volume knob for the CD player, I had not come up with a device that made use of more than one sensor in a related way. At some point, I’d like to look at the gyro/accelerometer combo.I just thought of a way to build an optical potentiometer using a photocell and two polarizing filters. Here are the components (from right):

  • A photocell is encased behind a polarizing filter, which I cut from RealD stereoscopic glasses (a circular polarizing filter).
  • Another polarizing filter is glued to a cardboard dial.
  • An LED, which I initially thought would serve as a constant light source for the device. But after implementing a test model, I determined ambient light was sufficient for demonstration.

Here is the assembled demo model:

oPP Proto

Here is a graph of the optical potentiometer’s output:

oPP Variable Resistor Graph

It gives good, clear differentiation as the dial is turned. But the range is a bit narrow. Perhaps this could be improved by encasing an LED over the top filter rather than using ambient light. I’m gonna keep working, more later…

No Comments

A trip down solid-state memory lane.

From Jules Verne and Isaac Asimov to Minority Report and Children of Men, science fiction has served as a sort of prep school for a society that is perpetually moving towards the technological future. To speculate now is to be ready for change later, it would seem. Yet, while science fiction has often predicted what sorts of devices will ultimately become commonplace tools of modern life, the genre seems to consistently miss the mark in depicting how those devices will fit into our daily lives. As I walked around Astoria today, I saw no head-mounted VR displays, but plenty of smartphones and the like, which more or less serve the same purpose of an augmented, information-at-your-fingertips reality. The billboards I passed didn’t scan my retina in order to customize themselves for my particular demographic, yet my Duane Reade receipt offered discounts that seemed to be based on my past purchases. If people want change, they seem to want it wrapped in the appearance of constancy. As a result, there has always been a discord between what is technically possible and what is acceptable to people. In one sense, we may say these lines move in tandem – as the possible advances, so does the acceptable. This is particularly true of the TV and telephone, as there seems to be little opposition to the ever-increasing portability of these devices. Yet, certain technological ideals, no matter when they are proposed, remain equally futuristic and alien. Will sub-dermal RFIDs ever happen on a grand scale? It is unlikely. Playing Nintendo DS and reading a book have very little in common, but the form of engagement that each requires is similar to that of the other. Both involve holding a small object in front of you, which can be put aside at any time should someone from the real world wish to address you. This similarity apparently makes Nintendo DS more like a book than like a VR headset. Surely the familiarity of the form cannot alone dictate that acceptability of a technological device. It is hard to imagine that taking your daily vitamins will ever include the daily RFID. For the consumer to cross the Threshold of Creepiness, the reward must be great. The idea that your home would turn on your favorite music as you reach the door is clearly not enough. But most of us would presumably swallow a tiny surgical robot if it were able to save our lives. What does this mean for us as technologists? Either create something truly important or make sure your only somewhat useful device looks and feels like a book, a stone tablet, dirt, or like anything that has been around centuries.

No Comments

Week 1 Lab Projects

In our first week of Physical Computing, we looked at two basic I/O mechanisms: the switch and the LED. To start, I set up some simple circuits with a voltage regulator, buttons and LEDs in preparation for the exciting prospect of using the Arduino as an intermediary between the input and output devices (a central processor, if you will). Controlling an LED from the Arduino for the first time was a grounding experience – perhaps like a lifelong vegetarian’s first attempt at farming. Finally, it is clear that peaches do not come from a can. Of course, once you’ve seeded a single LED, your inclination is to scale up to 2, 3, maybe even 6 LEDs. I made a 6 LED marquee with the following loop:

for (int i = 8; i <= 13; i++) {

digitalWrite(i, HIGH);


digitalWrite(i, LOW);


// Where 8 represents the pin # for the first LED and 13 represents the last.

Since the Duemilanove Arduino has only 14 digital pins, I realized my marquee device was limited to a maximum of that number of LEDs assuming that I didn’t need any of the pins for any other purpose. In thinking about the assignment of creating a digital combination lock, it occurred to me that one could reasonably think of a Wacom tablet (or digital pen device) as a comparable technology – they are both, in a sense, about painting by numbers. To build a functional digital pen device, you need both pen tablet and some form of display. With just a few pixels, I would soon surpass the number of digital pins on an Arduino. With some consideration, I realized that the power of a switching device could far surpass a 1-to-1 correspondence with an LED.With 14 digital input pins in 1-to-1 arrangement, there is a maximum of 14 inputs. With 14 digital input pins in a 2D grid, there is a maximum of 7×7 = 49 inputs. But we can do even better by using permutations. With 14 pins, we can achieve 2^7 = 16,384 inputs.Well, 16,000+ inputs would have meant a lot of wiring. So I decided to create a proof-of-concept with just 3 pins. This would allow a maximum of 2^3 = 8 inputs. I wrote out each possible permutation of 3 on/off switches and saw a problem: if one permutation is 0, 0, 0 (all three inputs off), can this permutation control an output? This would create a problem as how do you determine whether the user has pressed the 0, 0, 0 input button or simply pressed nothing? To get around this, I removed this permutation. Then, since I was left with 7 and I was trying to make a drawing tablet, I thought I should go to 6 just to keep things even. 6 inputs isn’t much, but if we can do it using only 3 Arduino digital inputs, then we’ve left some for other purposes (like output).So, as you might have guessed, I built a simple multiplexer and integrated it into a aluminum foil 6 pixel drawing tablet and 6 LED display array. The drawing tablet inputs are filtered through 3 input pins and though I could have reversed the multiplexing process for the outputs in hardware, I decided to just let those have a 1-to-1 correspondence with the Arduino pins and did the demultiplexing in the software using a series of conditionals.

6 to 3 Multiplexer Table

I attempted to create the multiplexer through simple hardwiring. Let’s look at B1 for instance. When the B1 circuit is closed, electricity is forwarded onto both M0 and M1. Seems right, but in practice there is a problem. B0-B5 are all effectively connected to one another through this hardwiring and so without correction, hitting any of B0-B5 would turn on all M0-M2. We need directionality! I did a little research and learned that a silicone diode was just the thing to achieve the directionality. But there was still a problem. Splitting B1’s current to M0 and M1 would mean less power at each M than was at B. So, I used an NPN transistor for each M so that even the reduced current reaching it would still trigger the proper state. In working on this project, the immense power of the seemingly simple switch became evident to me. At the time of this post, there were still some bugs in the system, which I believe are the product of a trial and error approach to certain technical challenges. I look forward to gaining a more formal knowledge of these components.

Week 1  - Multiplexer

No Comments