Sunday, October 18, 2015

Wine rack for charity

It's sometimes funny and interesting how these random projects of mine come together. About a month ago one of my wife's friends forgot her phone at our house. Despite it being late I offered to hop in the car and drive it over to her house. This turned out to be a great idea because her husband, a designer and fabricator, had scrap metal in the back of his work truck that night. He knows I have an addiction to making things and helped me transfer some round stock and plate steel into my truck. After letting it sit around my garage for a few days, I posted to Facebook asking for ideas about what to build.  I received a bunch of suggestions, but the one that was the most inspiring idea ended up being a wine rack for an upcoming charity auction. Here's a picture of the result. The project was donated to the Francis Towey memorial scholarship fund. The entire project was made from free materials.  The wood was milled by me from a willow tree that blew over in my neighbor's yard two years ago with the exception of the inlay which was a scrap piece of mahogany I picked up during a visit to North Carolina.

First step was taking the bar steel and turning it into circles. In order to do this I welded a pipe to a steel plate and then heated the bar with oxyacetylene and walked around the pipe.

Next, we used an abrasive saw to cut down the curled bar to separate them from the pipe.

Yes I'm wearing sandals. But I'm also wearing safety glasses. Eyes are more useful than toes, which also have more redundancy. 

Here are the rings after they have been cut and flattened.

After this, it was a matter of deciding what I wanted the rack to look like and just welding it together. 

I used my chainsaw and band saw to mill this willow two years ago. My neighbor loved this tree. When it started showing distress he even had an arborist come and try to save it. We were all sad when a few months later the roots pulled up and it fell over.

The wood was planed and jointed the edges prior to gluing two pieces together to make the top and shelf. I wanted to keep the live edge on the front of the top. In order to avoid marring the live edge with my clamps I used a scrap live edge which matched it and used that as my clamping surface. This will make more sense if  you look at the picture.

There are no biscuits in this joint. They are unnecessary for strength and I had planed on planing the boards one more time after the glue-up to ensure everything was flat and matched. The dark spot on the board is a low spot I saved for final planing. 

The night I did this glue-up was the night of the September 2015 lunar eclipse. Because I have two children, the only work I get done happens after bed time. I stepped outside of my garage and snapped this picture with my phone. My wife was standing beside me.  

My first attempt at an inlay ended poorly. The cnc drive belts were loose and it ended up skewing part of the design. It was my fault for being in a rush and not checking. I glued it all in anyway and filled the gaps with saw dust and wood glue. 

In order to hide the mistakes of the first attempt, I made the design larger and made a few practice runs with a high Z axis to make sure I would inlay over as much of the mistake as possible. This time everything lined up nicely. The finished project will look great. The inlay, more than just look pretty, covers up some gaps that were in the wood. If  you look at the pictures above from the glue-up you can see the imperfections I was attempting to fix.

Here things are starting to shape up.

For the front, I used Inkscape to design an svg that looked like a grape vine. The grapes, vines, and leaves are from three separate images which I cut apart, scaled, and arranged. I used the cnc to carve down 1/8th of an inch and then black spray painted the board. After letting the paint dry over night I ran it through the planer which took off a layer of wood and the unwanted paint. The only spray paint left was in my design. Perfect.

More welding to add brackets to hold the wood, some spray lacquer to shine up the wood, and spray paint on the metal and it was finished. It's hard to believe how well this boring white wood came to life after a few coats of lacquer. Between coats I rubbed the surface with 0000 steel wool which helped keep it smooth. For the metal I used rustoleum hammered black metal spray paint. This is an indoor project and the finish won't have to stand up to direct sun light or weather any way. I toyed with the idea of  using blueing on the steel, but the project was dragging on and I wasn't sure it would be a long lasting finish. I don't know who will be winning this wine rack at the auction and the chances are they won't know how to deal with or fix rust if the bluing were to fail. 

Sunday, September 13, 2015

Silicon casting to fix a Power Wheels

The final result, two new gears.
My nephew's Power Wheels jeep stopped working a while ago. We were able to track the problem down to a stripped gear causing the electronic differential to stop working.

Differentials, both mechanical and electronic, send power to the wheel with least resistance. This was useful in our case because it meant that the stripped gear would just spin and it's counterpart gear would receive no power. A positive transmission would have kept delivering power to the remaining gear and destroyed it as well. By having a good gear to reference, I was able to fabricate an exact copy by using a silicon mold resin casting process.

This was my first attempt at doing something like this. I'll also use this post as a place to follow up and capture lessons learned if this new gear ends up not lasting.

Making the mold was a fairly straightforward process. I hot glued the good gear to a piece of wood and then hot glued the top of a paper cup around it to hold the silicon while it cured. In the foreground is the gear and behind it is the vacuum chamber which is degassing the silicon mold material.

Before and after pouring the silicon mold material, I placed it in a vacuum chamber to degas. Here's a pictures of bubbles evacuating from the silicon. I mixed the binary liquid together using a wooden paddle and this always introduces air into the mix. Surrounding the mixture with a negative pressure causes this air to expand, float to the surface, and eventually leave the mixture. The material expands quite a bit in it's liquid form while this happens, so degassing before pouring ensures that the material doesn't overflow the mold reservoir and spill into the chamber. In the picture below you can see the "high water" mark of the expansion of the silicon before it settled back down to closer to it's original volume. I determined how much silicon mold material I needed by pouring dried rice into my mold form and then pouring it into a cup. I then put a mark on the cup and mixed material to that level.

After pouring the silicon into the mold form, degassing, and then waiting for it to react, this was the result. The form is a near perfect mold of the gear. Thanks to the degassing steps, there were no bubble pockets on the working surface of the mold.

The same degassing process was used for the epoxy resin I used to cast the actual new gear: degas in the mixing cup, and then degass again in the mold. The below picture is of the second degassing step, after I've poured the mixed epoxy into the casting mold. I wrapped duct tape around the edges of the silicon mold to create a reservoir. In the future, I think I'll mold a reservoir lip into the actual mold so I don't have to do this. The tape absolutely refused to stick to the silicon so this is mostly a pressure fit with the tape only sticking to itself.

I did not use any mold release spray for the first gear and it came out of the mold just fine. This gear will be ready to use after sanding the back flat on a belt sander.

Parts list and additional notes:
My vacuum chamber consisted of a 90$ Harbor Freight 2.5cfm automotive vacuum pump, 1/4" OD tubing, a cutoff valve, a wide section of steel pipe stuck to a flat piece of steel with silicon sealer, a piece of craft foam for a gasket, and some plexiglass.

The valve is necessary to sustain a vacuum. If you turn off the pump and leave it sit, the negative pressure in the chamber will start to suck oil from the pump and spray it into the chamber. My initial attempt at using an aluminium pot for the chamber resulted in crushing the pot, so be sure to test your chamber before putting anything important in it.

The blue gear is straight casting epoxy. The red gear I mixed in some sawdust. I'm not sure if the sawdust will add strength, but it did take up some of the volume and saved epoxy resin.

I was able to attain different colors by adding normal food coloring during the mixing step. Adding this color also ensured a uniform mix of the epoxy, once the drop of color was evenly distributed I would know the two transparent liquids were probably mixed thoroughly.

Sunday, May 31, 2015

Big giant table gardening

I've had a garden every year since 2009. The story has been the same every year. I try to grow food and then children, dogs, weeds, rabbits, ants, and squirrels conspire to ruin parts of it. But that's the game, if it was easy it wouldn't be worth doing. This year I think I've had a major technological breakthrough.

It all started by ripping a bar out of my basement. While trying to figure out what to do with all of the excess wood from that project, I ended up designing this garden box. It's put together using woodglue and 18guage pneumatic nails but designed so that if the glue and nails fail the forces exerted by the dirt in the box will still keep everything together. Here is my daughter helping me prove this concept.

At some point, I had the idea to put the garden boxes up on saw horses which kept them out of reach of our children and the dogs. This worked out so well that I built a giant table to hold them. Here's pictures from the build.

The table is made rigid by the the braces I placed along all three axis of the back leg joints. This worked out incredibly well and I think I'll use this design again in the future. I cut the legs so that the table would be level, which is why they are shorter on the far end. 

As shown above, I originally used pallets to hold the planters. It was only a day before I found enough time in the garage to cut all of the palet wood out and pneumatically nail them to the top of the table. A note about all of this: I left enough of a gap between all of the boards in this project to account for expansion during watering. I still expect to see failures, but everything is trivial to fix. 

Total cost of project was 26$, because I only had to pay for the 2x4s and 2x6x12 boards to build the table structure. Everything else was salvaged from my basement bar tear down or pallets that we received server shipments on at work.

It's way easier to tend to this garden than any I've ever had before. There are no ant nests in the soil, I don't have to bend over to pull weeds, the dogs and children ignore it, and I have a nice work surface I can keep tools like shovels and gloves just laying around on.

Wednesday, May 27, 2015

Lathe project

Some video and pictures from the latest project, an Atlas 10D 246 vertical drive lathe restoration. 

These guys have been awesome help so far: the mailing list has an extremely deep knowledge of these old tools and is eager to share information an dideas.

The ways determine accuracy of the lathe. The ways are the metal parts where the carriage moves toward and away from the jaws of the jaws of the chuck, which holds the spinning material. Older lathes wear here first and lose accuracy. Here I am trying to measure for problems.

On overall picture of the project in pieces.

The first real problem. A carriage gear is missing a tooth and the gear case is broken into three pieces. Unfortunately the case is an injection molded alloy metal with a very low melting point so welding or brazing are out of the question. The metal itself is an interesting mix of aluminum, zinc, magnesium, and copper called Zamak. 

 The gear will have to be pressed off the shaft, I have a new one on the way. Still researching how to best fix the case. That may take a while to think through. 

Update: Last night I presented this broken part to the CAMS club monthly meeting. I mean actual presentation. I stood in front of the group and described the project, it was a fun impromptu public speaking opportunity. The group discussion about the project was informative and I think I have several good approaches to this project.  

There were plenty of ideas about how to fix the broken part. I think I'm going to try alumaweld aluminum rods to do the repair. I picked some up today at harbor freight, 15$ for a ten pack, along with a cheap butane torch. While in Home Depot I also noticed that they had aluminum welding rods, so I picked some of those up $3.69 for a pack of three. I'll have to experiment to see which works better.

At the end of the meeting, one of the club members who was following my project on the mailing list pulled me aside and presented me with some better chucks for the lathe and sold them to me at a great price. The original three jaw chuck is in the picture to the right. The two chucks to the left are a better three jaw and an independant 4 jaw that I purchased at the CAMS meeting.

Update: Today, my new gear arrived in the mail. I spent a few minutes taking things apart and getting them ready for alumaweld. After some light heating with a small butane torch, the gear was easily tapped off of the shaft. 

I've since cleaned the oil/grease off of these, but neglected to take a picture. I'll post more after I attempt the weld.

Update: Weld failed. I heated the part slightly too much. Without any warning or indication it transformed from a part to a thousand little molten balls of metal skittering across my work bench. It wasn't that important anyway since the lathe is somewhat usable without it. I have plans to fabricate a new part from scratch, I can infer the dimensions from my sketches and the bolt holes on the lathe apron.

Thursday, May 21, 2015

i before e except after c

Somebody misspelled the word "weird" in an email to me this morning. Our HBase system was acting weird and they were sending me a note. "Weirding" is a common occurrence with big data software, so this was nothing surprising.

What caught my attention was that "weird" is another one of those words that violates the only grammar rule most people know.  We've all heard and memorized that "i comes before e except after c". Weird.

I believe that spell checkers have made us all dumber since we're able to outsource our thinking without really thinking about it. I've often found myself just hammering away at keys and letting the computer just generally figure out what I was trying to say. The computer is accurate and able to do do this, so we've formed sort of a symbiosis in this manner. But as a consequence I've found myself embarrassingly uncertain of my self when hand writing letters or notes with pen and paper. So I've tried to slow down and eschew spell checking systems before I become any more incompetent. Now I'm trying to pay attention to the spelling of words.

So how many words violate this rule?

Here's the wikipedia page:

If we scroll down to the Exceptions section we see four violations of the "cie" part of the rule listed. They are all words I'd never use, so that's not helpful and doesn't seem comprehensive. There's no real numbers anywhere in this article to look at. Maybe we can do better.

My next stop was here:
It was just the first page I found that had a list of english words. There are about 100,000 of them in a nice text file.

Grab the file

eric@glamdring:~/workspace/words$ wget

And count some stuff

eric@glamdring:~/workspace/words$ grep ie wordsEn.txt | wc -l
eric@glamdring:~/workspace/words$ grep ei wordsEn.txt | wc -l
eric@glamdring:~/workspace/words$ grep cei wordsEn.txt | wc -l
eric@glamdring:~/workspace/words$ grep cie wordsEn.txt | wc -l

Hang on a minute... "i before e, except after c". That's strange that there's more occurrences of "cie" (322) than there are of "cei" (88).

A quick look tells us why:
eric@glamdring:~/workspace/words$ grep cie wordsEn.txt | head

It looks like there are a lot of occurrences of a popular suffix "ies". A quick trip to the wikipedia page about suffixes.
So....... despite being used 6 times on the wikipedia page "ies" isn't listed as a suffix.  That's frustrating.

More searching and there's a page about it on wiktionary:

Let's filter those out.

eric@glamdring:~/workspace/words$ grep cie wordsEn.txt | grep -v cies | wc -l

Not bad. That's small enough of a list to take a look at. But I have a hunch "science" will show up a  bunch of times, since that's one of the exceptions I remember. And hey, we're not being very scientific anyway, so let's get rid of that too.

eric@glamdring:~/workspace/words$ grep cie wordsEn.txt | grep -v cies | grep -v science | grep -v scientific | wc -l

Not too many. Here's what's left.

eric@glamdring:~/workspace/words$ dos2unix wordsEn.txt
dos2unix: converting file wordsEn.txt to Unix format ...
eric@glamdring:~/workspace/words$ grep cie wordsEn.txt | grep -v cies | grep -v science | grep -v scientific | tr '\n' ' '
abortifacient ancien anciens ancient ancienter ancientest anciently ancientness ancients bioscientist boccie bouncier calefacient chancier coefficient coefficients concierge concierges conscientious conscientiously conscientiousness deficiency deficient deficiently delirifacient dicier efficiency efficient efficiently facie fancied fancier fanciers financier financiers fleecier flouncier geoscientist geoscientists glacier glaciered glaciers hacienda haciendas icier inefficiency inefficient inefficiently insufficiency insufficient insufficiently intersocietal jouncier juicier lacier lanciers liquefacient mincier nescient nescients objicient omniscient omnisciently overconscientious prescient pricier proficiency proficient proficiently racier saucier scientist scientistic scientists societal societies society specie spicier stupefacient sufficiency sufficient sufficiently unconscientious unconscientiously

Notice I had to use dos2unix. Windows and a few other programs really dork up newline characters, which makes a lot of transforms involving newlines not work. In this case I had to convert it so I could change newlines into spaces.

But back to weird... what's the deal with that category of exceptions.

Actually, nope. Times up. I'm done with my coffee and about to walk out to go to work, so that's where this post ends. 

Saturday, February 28, 2015

Ubuntu Dual Boot on Macbook Pro

This morning I dual booted a new Macbook Pro with OSX Yosemite and Ubuntu 14.04. These are my notes. I just stepped through this a few minutes ago on a brand new computer; it worked. YMMV

Ubuntu 14.04 AMD64 bootable USB drive.
I created mine using the StartupDiskCreator creator tool included with Ubuntu after downloading the latest .iso from

Step 1: Install rEFInd. 

Link here:
Make sure it persists across reboots before moving on to next step.

Step 2: Free up some space on the hard drive. 

With Yosemite I had to revert from a logical volume to a normal partition in order to be able to resize the OSX partition. On the command line run this.

diskutil cs list

You'll see your corestorage devices listed. One entry, the last one to print, will have this tag or something like it: "Revertible:  Yes"

Take the UUID from this entry and revert it by running a command like this:

diskutil cs revert 7BF42B7B-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Shut down completely and then power on the computer. Not sure why this is necessary, but it is.

You can now use the OSX disk tool to resize the partition by dragging the corner. Leave the rest of the space empty.

Step 3: Install Ubuntu

Plug in the USB stick. When you shutdown and restart the rEFInd will have it as an option to boot into.

Step through the install screens as normal and choose to "do something else" during the disk selection step.

Add two partitons, one ext4 for root (/) and one as swap.

This is important: make sure to install the bootloader on the same partition you chose for root (/). This is available as a dropdown menu option on the partitioning screen.

Finish the install, the next time you reboot you'll see the OSX logo and a Penguin in your rEFInd boot menu. That's it.

Other notes

To get the computer to sleep properly, install the proprietary gpu drivers. This is a one click install under the "additional drivers" tool.

To right click, use two fingers while clicking.

I've found it useful to disable tap-to-click on the touchpad. This checkbox is in the mouse settings tool.

Wednesday, February 25, 2015

Praise punishment and regression to the "mean"

Using punishment as an incentive is wrong-headed.

First, let me put into context a few life events that have shaped my thinking on leadership and training. I enlisted in the United States Marine Corps when I was 17 years old. Maybe I'll go into details on why in a later post, but a few months after my 18th birthday I was on a bus to Parris Island, South Carolina. They organize transportation so that the bus arrives on the island at 3 a.m. then all hell breaks loose for the next 12 weeks; this is known as Marine Corps Boot Camp. A few years later, I was selected for a scholarship and ended up going to a private military school, Norwich University. That in itself is another story for a different time (my friends reading this that have heard the story are laughing right now). The first year attendees of Norwich go through an extended bootcamp known as Rookdom. A few years later I found myself in Quantico Virginia attending USMC Officer Candidate School, which followed an 8 month training program known as "Bulldog"; yea that was fun. A year after that, I spent six months at USMC The Basic School, which effectively is another boot camp, but with more emphasis on peer leadership. Few would argue that I've had more than my fair share of training that has traditionally made extensive use of punishment as an incentive. Based on my assertion in the first sentence of this post you might think that I had a tough time in these schools. But the truth is that I didn't; actually I excelled and enjoyed all the hard training. My first year in the Marine Corps I did so well that I earned two meritorious promotions, those successes lead into my selection for the officer candidate programs that eventually gave me the opportunities to be commissioned as a Marine Corps Officer and spend five amazing years leading Marines. You'd think having such a great experiences with punishment as incentive that I'd be all for it and giving everyone I work with a healthy dose of Drill Instructor type "constructive criticism". Those that work with me will tell you that nothing could be further from the truth.    

Steven Pressfield's excellent book "The War of Art" has a few sentences about the Marine Corps that I enjoyed.
    "There's a myth that Marine training turns baby-faced recruits into bloodthirsty killers. Trust me, the Marine Corps is not that efficient. What it does teach, however, is a lot more useful. The Marine Corps teaches you how to be miserable. This is invaluable for an artist. Marines love to be miserable. Marines derive a perverse satisfaction in having colder chow, crapper equipment, and higher casualty rates than any outfit of dog faces, swab jockeys, or fly boys, all of whom they despise. Why? Because these candy-asses don't know how to be miserable."

This puts all of the pain and abuse of "corrective" training in the Marine Corps into context in my mind. Sure, part of it is an incentive to drive yourself into the ground in the pursuit of excellence. But even those that did well still caught the abuse. The traditions of negative training in military contexts is to breed mental and physical toughness through adversity. Having experienced it all first-hand I can assure you it is not conducive to other environments.

I was walking through a restaurant several weeks ago and in a side corridor noticed a manager berating one of the waitresses over some minor offense. I don't even know what it was, but they stopped after they saw me walk up. Of course I didn't say anything, it was only a minor discussion and none of my business. But I wish I could have had a conversation with the manager and explained to them that punishment only breeds contempt, not better performance. The evidence for this is so overwhelming that you'd think anyone in a leadership position would see it as obvious. But there's an illusion at play in these scenarios that makes unobservant leaders think that punishment works.

Let's say I'm a new leader trying to figure out how to do performance management and the two tools I'm learning how to use are negative and positive reinforcement and I employ them both. The first thing I try (because I'm a good person and I want to praise people) is praise someone for doing well. They met a deadline, mopped a floor, sold something, or wrote exceptional code. Great! Happiness and celebration all around. But I notice that the person I praised usually doesn't do as well the next time. The next deadline might be missed, they missed a spot mopping, missed a sale, or the code has a bug.  What gives? Ok, next I'll try negative reinforcement. So somebody under performs and I let them have it. I give them bad performance appraisals, mention their failure, give them demerits, counsellings, or whatever. Over time, I start to notice that more often than not, when I've punished their poor performance they seem to get better! I'm not making this up! In my experience this is exactly how things work. So why wouldn't I advocate for using punishment to increase performance? Because all of this is just an illusion.

Think of performance as random and happening across a range of potential outcomes. Like a stock chart it goes up and down, so goes an individuals performance at a task. The overall trend might be up, or it might be down, and this is the real thing to watch. But look at the chart and see where we would notionally apply positive or negative reinforcement. If we apply positive reinforcement at the peaks, statistically speaking the next thing that happens is going to be less stellar than what we praised. If we apply negative reinforcement at the troughs, statistically the next task is going to be better. This happens in both scenarios regardless of whether our intervention had any impact at all leading to the illusion that negative reinforcement increases performance. It's a bias that needs to be overcome. In statistics and sociology this concept is called regression to the mean and it's a well studied phenomenon which can be summarized as such: "if a variable is extreme on its first measurement, it will tend to be closer to the average on its second measurement".

So the next time you find yourself intuitively thinking that "punishment works"; take a step back and remind yourself of this illusion. It really does pay to be good to everyone, even though at a superficial level it doesn't seem to work as well.

Wednesday, February 18, 2015

Small fans suck

It's 3am on a Monday morning. My son Colton was up because he hasn't been feeling well. My amazing wife managed to get him back to sleep after some tender loving care, seriously she is awesome. But I'm up for the day; so I wandered downstairs, made some coffee, and got on my computer to get some work done. As soon as I sat down, the serene silence of my morning started being cut to shreds by the buzzing of a fan coming from my computer. It had started several days ago but I haven't had the time to fix. Next thing I know, I'm shaving a  yak.

First some background before I get into detail about how I screwed up and what I did to fix it. A while ago I spent several weeks being obsessive about the noises coming from my computer. It took a surprising amount of effort to get everything down to a tolerable level. I eventually got it so quiet that I moved my cpu tower onto my desk next to my monitors (and out the reach of toddler fingers).

For those of you wondering, here's my recipe for a quiet PC. First, get rid of any small fans in your case. They  Replace them with bigger fans, then run those larger fans at a slower RPM. Fan speeds can be tuned in your computer's BIOS, so hit that F2 (or whatever) button on boot and get in there to tune them down. Run them as slow as you can. Your computer will shut itself off if it gets too hot, don't worry about it. Computers are not people, they have a different temperature range they can run safely at. If you absolutely think your computer is too hot, add more big fans running at the slower RPM. After a lot of research and chatting with friends, I settled on the Cougar quiet fan listed below. It has a good blade design and rubber grommets where the screws connect it to the case.

Next, get rid of the default CPU heat sink and fan. I went with liquid cooling at first, but since you have to run a fan over the radiator this doesn't have any benefit over just getting a larger heat sink. After I had one liquid cooling unit stop working I actually eventually settled on just a big heat sink to replace it, which was also way cheaper. I replaced all fans with the large Cougar fan I mentioned above, the default fans on everything were noticeably louder. I like passive heat sinks that allow me to move air parallel to the board. This allows me to arrange the fans to move air through the case in a coherent way rather than creating eddies. Here's links to the hardware I'm talking about:

I had it good for while with this setup, then I messed it up; here's how. This is a picture of the front of my computer for context. Sorry for the low quality, it's dim in my office right now.

I've had this case for over ten years, it's an Antec 1080 and it's been great. I splurged at the time, but it ended up being worth it. After a few years of strictly console gaming (7ish) I've started using my PC to play games again. This is mostly because my PC is in my office and I'm not comfortable playing most video games in front of my children yet. The content of most modern video games is perfect fodder for children's nightmares and I don't want anything to do with that. Now, I run Linux for most everything, but occasionally there's a game not available and I have to use Windows. For a long time I just partitioned my drives and dual booted to accomplish this. But this solution is rife with problems that I don't want to deal with. So, I bought an extra SSD drive and a 4 bay hot swap device. It's the thing in the picture with the four red clips. I only have one of the drive bays actually connected to the computer (top left); the other three are just holding slots. With this setup I can have four different operating systems and when I want to switch I just shutdown the computer and switch the drive I want to use to the active bay and power up. It's been super convenient, so much so that I bought a larger hot swap bay to use larger drives. The larger hot swap bay is second from the top, just under the optical drive. With the larger bay I went with a trayless one. This way I can grab any old sata hard drive and just shove it in there without having to worry about screws or buying extra bays.

The only downside of these devices is, you guessed it, they come with integrated tiny fans. When first installed, the fans ran quiet enough and things were alright. But only a few months later and the little fan turned into a buzzing whining nuisance. Never trust a small fan. Thirty minutes with a screw driver and I liberated the nuisance from my computer case. Here's a picture of the scoundrel.

My computer is back to it's normal quiet self. Thanks to my big fans pushing air around my case I don't expect to have any drive heating issues. If I do I'll post a follow-up here. Yak = shaved. Now, I'm back to work.

Sunday, February 8, 2015

Internet knowledge

The exchange of knowledge happening via the internet is making everything faster and better. Even old skills like knife making.

"Much of this energy is relatively new. “When I first got into this business, in 1968, I had a hard time finding fifteen knifemakers from Alaska to Florida,” A.G. Russell, the ascot-wearing don of the modern knife market, told me. “I’ve got three thousand in my computer file now.” Nearly everyone credits much of this explosion to the Internet, which not only has made heretofore obscure items suddenly accessible, but also has spread knowledge about the craft behind these items to a younger generation. “The guys just starting out today, their knives are as good as the best makers’ fifteen to twenty years ago,” Steve Shackleford, Blade’s longtime editor, told me. "

Tuesday, February 3, 2015

Big data benchmarks and High Performance Computing protocols

I was asked today about benchmarking between high performance computing (HPC) and big data systems. In the context, the discussion was about different software architectures deployed across commodity hardware; not the specialized systems you see on the top500 list. To clarify: supercomputers are something you buy, high performance computing (HPC) is something you do. Don't get them mixed up. Not very many people play in both areas, the few that do don't seem to think very empirically about both; so there's not much concrete information go by. This makes the question a difficult one, but this was my best shot at answering it at the moment.

My immediate response was that the lighter weight, less fault tolerant, HPC architectures will always be faster given the same number of resources, but they are of limited use for big data. Benchmarks comparing big data and HPC architectures to each other don't exist in a meaningful way because the benchmark algorithms used are different. Big data's most prominent benchmarks are sorting problems which operate on commonly available data. HPC's most prominent benchmarks are algorithmic, for example Linpack TPP, which is a system of linear equations (i.e. math stuff with no data).

In my own experiences, heterogeneous compute environments (aka GPUs) typically have used High Performance Computing protocols, like Message Passing Interface (MPI) based applications. This is because the jobs they are used for don't take a lot of time (right now at least). HPC protocols like MPI are notoriously bad at fault tolerance, so software or hardware failures cause the entire job to be restarted. We use HPC protocols because there's lots of legacy code written for them and it keeps the jobs fast, starting over isn't much if a big deal because the jobs typically run so fast anyway.

Restarts on the big data jobs are more of a problem because the jobs often run for much longer, meaning lots of wasted time if you have to go back to the beginning. A compute job running over 10,000 compute hours is almost certain to have a hardware failure. Software failures are much more common (locking conditions, out of memory, segmentation faults, bad data, a million other ways to fail).  Because of this, big data software is more fault tolerant. This makes it slower, but able to do those long running jobs on big data. Big data software has been getting drastically faster, but at the cost of using more computers simultaneously to attain that speed.

Lots of people have attempted to benchmark performance across the different systems (which are myriad in both of these categories I've mentioned here). But none of the benchmark standards have seen very wide adoption. This is because they are like the systems they're trying to benchmark: new, with plenty of room to get better. The fundamentals keep changing. In my mind this speaks to how new all of this stuff really still is. Everyone is still trying to find the best way to do things and making massive progress. It's an exciting time to be working with these technologies.

Some good benchmarks and graphics comparing the various platforms: