Saturday, December 13, 2014

Baby monitor antenna fix

Our three year old son woke up early one morning this week. Before my wife and I were able to wake up, he had already somehow snapped the antenna casing off of his sister's baby monitor on the nightstand.

Through the course of the morning, the casing became lost in the shuffle. It's either in the garbage or somewhere in the house behind a heavy piece of furniture. I waited a few days to see if it would  turn up. For all I know it's on another planet by now. This morning I decided to make an attempt at covering up the fragile antenna wire. Some hot glue and an old pen seemed to do the trick. Here's a video of the fix:

Pretty much anything would have worked fine for fixing this, as long as it didn't conduct electricity. Conductive materials would attenuate the radio signal and decrease the range of the device. I had even considered hot gluing the antenna wire to a piece of wood, but I think this solution will make my wife happier.

This is the second time I've written about fixing baby monitors. In my other post about a different baby monitor I fixed a bad power port. The first fix was much tougher than this cheap pen trick.

If you've been following my videos or blog for a while, you might notice the antenna wire looks like a tiny coaxial co-linear. The size of the segments looks about right for the higher frequencies these monitors use. Here's an old video of me building a co-co antenna to capture ADS-B traffic. That was a fun project that worked, but I never ended up writing much about it. There's already great resources on the Internet about that if  you're interested in ADS-B and I didn't add much to what others have done.

Tuesday, December 9, 2014

Building strings in Python

The Saturday morning Python Christmas tree was fun and brought up a few interesting questions. A friend, Chris, made the comment that building strings as a list and then joining is more efficient than concatenating as you go along (like I did). I'm sitting in a room waiting for new tires to be placed on my truck, so this is a great little problem to take a look at.

Here's the code in question:

We build each line of the tree by concatenating strings together. This is a common thing to do and worthy of closer examination.

Here's an example of what I did in the Christmas tree code:
string = "" # start with a blank string
for _ in xrange(i): # make a loop
    string += " " # for each iteration of the loop, redefine the string

Here's what I think Chris recommended:
appendlist = [] # start with an empty list
for _ in xrange(i): # make a loop
  appendlist.append(" ") # append to the list
string = string.join(appendlist) # join the list into a string

As a first step, I wrote each of these and timed them using the Linux time command. By just picking random sizes of strings and timing the results, it seemed like my version was faster.

I then used Python's cProfile module to take a closer look, but wasn't able to discover anything useful with it. Plotting the two methods over a range of string lengths seemed like a logical next step and ended up making things much more clear. With a timing decorator that wrote results to csv files and another loop to iterate over a range of string lengths, I was able to create the data to make this chart:

It appears that "append as you go" does outperform "build a list" up until a point. The erratic behavior of appending to larger strings probably has to do with how memory is allocate for strings vs lists. Lists are made to grow smoothly, while strings are more difficult to allocate memory for because growing them in this manner isn't something that happens often. Even though strings can be abstracted as a list of characters, they aren't the same thing according to this. All that said, if your concatenating a string that's under 6 million characters (even on my little XPS12 laptop), it seems there's no real benefit to either method.

The work on my truck is complete, so I have to end this post. Maybe I'll pick this back up some other time and try it on different computers. If you end up looking at this, share your code or thoughts on the github repository.

Here's the code I used to generate the data for the chart which I made with OpenOffice Calc:

(thanks reddit user dangerbird2)
The better performance of += operations can be explained by optimizations in the CPython compiler. This might not hold true for other compilers (but probably does). However, the Python Style guidelines recommend against relying on compiler specific optimizations. So it's still good form to use list based string concatenation.

There's also a great reference to an article by Spolsky with a joke about a painter that illustrates the problem we'd expect to see by concatenating strings poorly.

Here's the article:

Here's the joke:
This code uses the Shlemiel the painter's algorithm. Who is Shlemiel? He's the guy in this joke:
Shlemiel gets a job as a street painter, painting the dotted lines down the middle of the road. On the first day he takes a can of paint out to the road and finishes 300 yards of the road. "That's pretty good!" says his boss, "you're a fast worker!" and pays him a kopeck.
The next day Shlemiel only gets 150 yards done. "Well, that's not nearly as good as yesterday, but you're still a fast worker. 150 yards is respectable," and pays him a kopeck.
The next day Shlemiel paints 30 yards of the road. "Only 30!" shouts his boss. "That's unacceptable! On the first day you did ten times that much work! What's going on?"
"I can't help it," says Shlemiel. "Every day I get farther and farther away from the paint can!"
If you ever hear the term "Schlemiel the Painter algorithms" this is what is being referenced.

Sunday, December 7, 2014

Leather axe sheath

Tonight I made a sheath for the axe I posted about a few days ago. Here is a picture of the completed project.

Below is a list of supplies to do this type of leather work. It's fairly straight forward once you have the basic tools and doesn't take very long for small projects like this.

Here's the steps I took:

  1. Draw your design on the leather with a pencil and cut it out.
  2. Use a hollow punch to make a hole for the first rivet. Secure the first rivet before doing the rest, this will keep everything aligned.
  3. Draw a line where you want the stitches, then follow the line with the leather stitch hole punchers.
  4. Follow the directions include with the sewing awl to insert the threads and tie it off.
  5. Place grommets.

Some lessons learned:

  • Placing the rivets before sewing was a good idea. It would be difficult to keep everything straight without them.
  • Wherever possible, leave extra leather so that the pieces can be trimmed together after the project is complete.

Although I've horded leather scraps for years, I used this stuff from Amazon and it worked fine.

Leather stitch hole punchers (my name for them). They look like this:

A sewing awl. They usually come with string.

Some small rivets.

Small gromets.

A small hollow punch set, like this. I got mine at Harbor Freight for slightly less, quality of steel isn't a big deal since we're just punching leather.

Oil to protect the leather.


Saturday, December 6, 2014

Python Christmas tree

I woke up this morning and meant to clean my office. But I ended up drinking coffee and writing this program to avoid doing that.

It works, and I learned how to record my desktop to show an animation I created by looping it in a bash command. But there are several places it can be optimized. Can you spot them? Comment on the youtube video if see the places for improvement.

# Prints a christmas tree on the terminal made up of ascending and descending integers
# 2014
import sys
import random

size = int(sys.argv[1]) # first argument is an integer which is the height of the tree
probability_green = .7 # how much of the tree will be green vs a random ornament

class colors: # colors
    purple = '\033[95m'
    blue = '\033[94m'
    green = '\033[92m'
    yellow = '\033[93m'
    red = '\033[91m'
    end = '\033[0m'
ornaments = [colors.purple,, colors.yellow,, '']
def decorate(s):
  if random.random() > probability_green:
    color = random.choice(ornaments)
    return color + str(s) + colors.end
    return + str(s) + colors.end
for i in range(1,size+1):
  line = ""
  for s in range(i, size): # this will loop over each digit left
    for c in range(0,len(str(s))): # prepend a space for each character of each digit
      line += " "
  for j in range(1,i+1):
    line += decorate(j)
  for j in range(i-1,0,-1):
    line += decorate(j)
  print line
trunk_length = 3 #TODO: if this wasn't a joke I'd make this an argument
for i in range(0, trunk_length):
  line = ""
  for s in range(1, size): # this will loop over each digit left
    for c in range(0,len(str(s))): # prepend a space for each character of each digit
      line += " "
  print line + "00"


 Getting in the spirit, a friend (Sage) sent me this bash version a few hours after I posted this.

to run: ./ 10

declare -a a=('.' '~' "'" 'O' "'" '~' '.' '*')
[[ $# = 0 ]] && s=9 || s=$1
[[ $s -gt 5 ]] || s=5
for (( w=1, r=7, n=1; n<=$s; n++ )) ; do
  for (( i=$s-n; i>0; i-- )) ;  do
    echo -n " "
  for (( i=1; i<=w; i++ )) ; do
    echo -n "${a[r]}"
    [[ $r -gt 5 ]] && r=0 || r=$r+1
  echo " "
echo " "

Update 2:
A new post to follow up on concatenating strings in python:

Sunday, November 30, 2014

Another axe handle

I found an axe head that was just a little too large for any of the pre-fabricated axe handles I could find for sale at hardware stores. I was committed to saving it from the scrap pile as soon as I saw it, but it's been sitting in my garage for the better part of a year for lack of the right piece of wood. I was recently able to find a satisfactory piece of hickory to make a new handle from and accomplished the task over two days this last week. Here are some pictures of the build.

Of course I took it out immediately and started hammering on a fallen tree. I was confident in my craftsmanship, but it felt good to just feel the solid thunk of the axe into the wood and feel that the shape and fit were as good as they looked during construction. In one of the pictures is a previous axe I restored with about an overall weight of 5lbs, it feels like a feather compared to this new monster. This new axe is HEAVY coming in closer to 8lb overall weight with the much larger head and thicker handle. It is a lot of steel and wood to swing through the air. I imagine it will be great for splitting. Not that I get much use out of my outdoor tools where I live, but it's nice to bring tools like this back to life. Something tangible to hold up as a completed project always feels great.

When I brought the axe home I put on the first coat of boiled linseed oil. There were other oils at the shop, but it was worth waiting the few days to do the right thing.

Update: A few weeks later I made a leather sheath for this axe. Post here

Saturday, November 29, 2014

Mean people fail (2014)

Children and technology

Parenting rant.

Some of the junk people are writing about children and access technology are perfect examples of why activists should seek consult before interpreting scientific research. Contradictory research findings by an army of grad students studying the soft (pseudo) sciences are not a basis for the sound raising of children. Why do I say pseudo science? Because I've always enjoyed and agreed with this Feynman rant.

The "studies" linking access to technology to this malady or that malady and the articles attempting to summarize or use them as a call to action are just silly. I don't believe they are malicious, just ignorant and misguided. I refuse to even link to them here because I don't want people receiving attention for this type of behavior. We can only speculate as to how they can divine and attach to such rigid certainty about their conclusions that they are ready to scream for legislation or world wide bans on this or that random technology to x random threshold. 

Plausibility alone is not a criterion for accepting speculative tales, but enough people just run with this to make it perpetuating. Here's a classic paper on the topic of something called adaptationism in biology. To paraphrase the topic of the paper, they poke fun at biologists that identify some trait of an organism and jump to conclusions about why it makes sense from an evolutionary perspective. The problem becomes worse when considering phenomenon like citogenesis. Randall Munroe explains:

Now, that said, I certainly monitor what my young children watch (avoiding violence and media that I feel teaches the wrong lessons), but I'm excited to see him grow up with technology I didn't have. Already he amazes me every day with his aptitude for taking advantage of the tools and what he pays attention to as he tries to wrap his brain around bigger concepts. Every day we have thoughtful conversations that are really amplified by the other things he is learning. I'll be getting him his own tablet as soon as he is responsible enough. I don't know when that is yet, but probably soon. Until then I don't deny him access to devices unless there are externalities such as bed time, dinner, etc.

So where do parents go wrong with technology? I suspect, as with many things it's complacency. We are all guilty of it to some extent, whether it's because we get lazy or are focusing on something else like our careers. You'll hear terms like "the electronic babysitter". This is funny (and, honestly, sometimes true), but all too often I believe people just throw the towel in and give up. Ignoring the children's development or blaming your conflicts with them on factors out of your control are recipes for disaster. I believe that to be a good parent you can't make excuses or give up. You constantly need to deliberately detect, approach, and overcome where you might be going wrong. But first and foremost you need to let the child know you love and care for them and that you're acting in their interest. Never give up and have the humility to adjust when you might have been wrong. ...oh, and make them laugh while you mentor them. Approaching with love really means with love, not an iron fist.

That's the end of this rant. I'll come back to it in a few years and see if I feel the same way.

Thursday, November 6, 2014

Creating alerts

This article has good advice for creating failure or event alerts and how to keep yourself sane.

"My Philosophy on Alerting based my observations while I was a Site Reliability Engineer at Google"

It's written by a Google Site Reliability Engineer. I have a few friends that have had this job in the past and I've really come to respect the position. Here's a random interview I found while trying to find a link to describe the SRE job.

Sunday, August 24, 2014

Fish inlay trivets or small cutting boards

This last week I had a chance to visit my wife's grandfather's wood shop and work for a morning. I decided to try an experiment with wood from his scrap bin and this is the result.

I started with a piece of 1" maple board 20" long which I cut roughly in half to make two pieces. Then I hand drew a fish made of two smooth arcs on each piece. I cut one of the arcs as smoothly as I could on the band saw. Smooth and straight was more important than accurate, so I drifted a bit from my original lines rather than making sharp corrections; this was important. It's critical to have a smooth curve right off the cut so everything fits together well.

On the table saw I ripped some thin strips of purple heart, roughly 1/8" thick and a little wider than my maple boards were thick (so just over 1"). I briefly soaked the purple heart in water and then did a test fit by clamping between the two pieces of maple I cut on the band saw. The purple heart didn't crack, so I repeated the clamping process with lots of glue and more clamps as tight as I could get them.

While one piece was drying, I cut the arc on the other piece and did the test fit and glue up. By this time the first arc on the first piece was dry enough. I took a hand plane and cut the purple heart inlay flush with the face of the maple and trimmed the excess ends off on the bad saw.

It wasn't very pretty at this stage, the maple was covered in wood glue and wasn't clamped together perfectly straight, but it was flat enough to be stable when I cut the second arc on the band saw to complete the fish design. I then glued it back together with the same process and purple heart inlay. When I glued the second inlay I referenced off where the pieces cross near the tail of the fish. I didn't care if the edges of the board lined up (they didn't), but I wanted this little section to all line up. My inlay was wider than the kerf of the band saw, so I couldn't have more intersections of the inlays and actually have them line up. That's why the mouth of each fish is open, it wouldn't line up properly without a much thinner inlay to match the thin kerf of the band saw.

With the final glue-up dry, I squared it off using the mitre saw and table saw. Ran it through the planer to clean the faces off. I could have sanded them flat, but why bother with a 220v 15" planer handy! Rounded the corners off with the router. Then finally sanded a bit to clean it up. I coated them with some food safe mineral oil since I plan to use them in the kitchen.

This was beach week vacation and I had been drinking corona light on the beach and while fishing. I intended these as small cutting boards for limes or cheese. I gave one to my wife's grandfather and the other I took home to use in my kitchen. They will work just as well as Trivets. Since the wood goes all the way through, it would be trivial to re-sand and finish them if they were ever to get burnt or scuffed.

When I get home, I plan to use some dark exotic hardwood and maple inlays to make an inverse board to compliment this one. I have the wood to do it now, I took a pile scraps home with me to work with. I plan to do a future post showing the cuts and glue-ups I described above.

I used this cutting board conditioner to protect them.


Monday, July 28, 2014

Creating users in a deployment script

The simple and lazy thing to do when creating a user in a deployment script is to throw a plain text password in the script. Avoid this temptation.

Here's a better way, which generates a random password and stores it in /root/ of the provisioned machine in case you need it. The major problem this avoids is that this software can now be safely made public or stored on github without concern to exposing credentials.

# create user
sudo apt-get -y install makepasswd
PASSWORD=`cat /dev/urandom | head -n 1 | base64 | fold -w 10 | head -n 1`
echo $PASSWORD | sudo tee /root/tangelo_password.txt
passhash=$(sudo makepasswd --clearfrom=/root/tangelo_password.txt --crypt-md5 |awk '{print $2}')
sudo useradd theusername -m -p $passhash
sudo usermod -s /bin/bash theusername
# end create user

Saturday, July 19, 2014

Floating shelves

This weekend's project was floating shelves. My wife found inspiration for these on pinterest. I made two sets of three, one set for the living room and one set for our bedroom. Here are pictures of the end results.

The construction was fairly straightforward table saw work. The edges were cut from a normal pine 2x4 you can get at the big box stores for 2$. I used two of them for this project. The top and bottom was made from 1/4 inch sanded plywood. I used a 4x4 ft section of plywood for all of these shelves and had some left over. The shelves vary in length but are all 6 inches deep from wall to edge.

I cut the 2x4s roughly to length on the table saw, then used my joiner on two sides of each piece to make them straight, then ripped it on the table saw to size using the joined edges as reference. Here's a video from somebody else (The Wood Whisperer) on how and why to do that.

Before cutting the corners to fit together, I cut the rabbets for the plywood with two cuts on the table saw. The rabbets are the grooves that allow the plywood to sit flush with the top of the edge board.

I then mitered the corners to fit together at 45 degrees, cut the plywood to fit for each shelf, and then glued them together. The plywood was glued everywhere it touches the edges. The miter corners are also glued together. Instead of clamps I used a pneumatic brad nailer and 5/8 inch 18 gauge brads on the top and bottom. I did not shoot any nails into the corners because I wanted to do the final bevel and routing after assembly.

After the glue set (an hour or so) I cut the edge bevel and ran the router along the bottom edge. The brad holes and any other gaps were filled in with wood putty and then sanded. If I was going to stain these instead of painting them, I probably would have used clamps instead of brad nails to avoid the holes and putty.  Here are pictures of how the plywood sits in the rabbets.

I used a micrometer to measure for brackets to hold the shelves to the wall. The brackets are just a chunk of wood that fits exactly inside the shelves top to bottom. I cut them to have some slack left to right to make assembly easier. When the shelves are installed these are completely hidden. Here is a picture of a bracket fitting inside of the shelf.

I took the brackets, a pencil, and a stud finder into the house where the shelves would hang. I marked the stud locations on the brackets then pre-drilled and counter-sunk some holes in them. Since the brackets are hidden I was free to use any size screw or bolt to secure them to the wall. Three inch deck screws into the wall studs worked fine for me. Drywall expansion fasteners would work great too.

When I screwed the brackets to the wall, I made sure they were level.

Then it was just a matter of sliding the painted shelf onto the bracket, pre-drilling a hole, and sinking a screw to hold them on.

With the brackets solidly mounted to the wall, I think these shelves will break apart before they ever droop or fall down. I hate droopy floating shelves and like this system more than the floating shelves you can buy. If I really want it to look clean, I might paint the screws, but for now they'll be hidden behind picture frames. The overall cost of construction was less than 5$ per shelf and all six only took three or so hours total construction time.

Monday, July 7, 2014

curation is not preparation

I was on an email thread where somebody mentioned spending 80% of their time on data curation. I wrote this in response.

I think there's a semantic error we're drifting into. I think the original thread below mixes up the activity of curation and data preparation. Data prep does take a lot of time, but curation is a different thing. I'll try to describe what I'm thinking here. Curation is more of an act of protecting integrity for later use, analysis, or reproducibility. Like a museum curates. The tools for data preparation are awesome and very useful, but move beyond what is just a curation activity which I'd describe as something more fundamental.

data curation != data preparation

Provenance is always paramount when making decisions off of data. That's why trustworthy persistent storage of the raw data is most important. Any transform should not only be documented but should be reproducible.  Canonical elements of a curation system would be the raw data storage (original formats and as attained), transform code, and documentation.

Enabling discovery (locating, standardized access protocols, etc...) starts getting into something beyond curation. The connotation of curation implies preserving the scientific integrity of the data. Like dinosaur bones or artifacts being curated in a museum. Some of them are on display (available via "standardized protocols" on display), but the rest are tucked away safely in a manner that doesn't taint later analysis of them. More often than not, the bones on display are actually semi-faithful reproductions of the artifacts rather than the original artifact. Same thing with data. The graph visualization (or whatever) of the data might not be technically still the same data (different format, projection, normalized, transforms, indexes, geocoding, etc...) but it's a faithful reproduction of it that we put in the display case to educate others about the data. Like a fiberglass T-rex skull tells us a lot about the real thing, it's not meant for nuanced scientific analysis. All transforms of data, especially big data, contain an element of risk and loss of fidelity which would taint later analysis. We're all so bad at transforms that we avoid using them in cases where a life is at risk (court proceedings or military intelligence analysis) processes require citation of raw data sets. A geocoding API rarely assigns location with 100% accuracy (it's usually an exception when they do), sometimes when we do things like normalize phone numbers there's an edge case that the regular expressions don't account for, things can go wrong in an unlimited number of ways (and have... I've caught citogenesis happening within military intelligence analysis several times). The only way to spot these problems later and preserve integrity of the data is to store it in it's most raw form. If we wish to provide access for others to a projection they want to build off, the best way to do it would be to share the raw, the transform code, and a document showing the steps to get to the projection. In the email below this behavior of later analysts/scientists is noted (with disdain?). It shouldn't take long to look at previous transforms and reproduce the results, if it does then those transforms weren't that reliable anyway. If those receiving the data just want to look at a plastic dinosaur skull to get an idea of it's size and gape in wonder, then sharing data projections (raw data that has undergone a transform) is fine.

When providing curated data for research or analysis, I even make it a point to keep a "control copy" of the data in an inaccessible location. That way if there is a significant finding there is a reliable way to determine that it's not because the data became tainted during an inadvertent write operation on it.

On the other end of the spectrum (which I see all the time) is the unmarked hard drive with random duplicative junk dumped on it as an afterthought to whatever project the data originated from. Although "no" is never an answer when handed something like this and certainly useful analysis can be achieved, this is below the minimum. I imagine it's like being handed evidence for a murder case in a plastic shopping bag with the previous investigator's half-eaten burrito smeared all over it.  You can sometimes make the case for a decision with it, but it's not easy and it's a dirty job trying to make sense of the mess. This is probably more the norm when it comes to "sharing" curated data in government and industry. It's ugly.

The minimum set of things needed for useful data curation:
1. raw data store (just the facts, stored in a high integrity manner)
2. revision control for transform code (transform code, applications, etc...)
3. documentation (how to use transforms, why, provenance information)

Everything beyond this could certainly be useful (enabling easier transforms, discovery APIs, web services, file transfer protocol interfaces), but is beyond the minimum for curation. Without these three basic things, useful curation breaks.

Sunday, June 22, 2014

Entomological enclosure (aka "bug house")

I took my son fishing Saturday morning. We didn't catch anything, but Colton enjoyed looking for bugs around the pond. When we got back to the house it was time for his afternoon nap. After he was settled down I wandered out to the garage and made this:

It's just a piece of screen and some scrap lumber. I cut everything to size on the table saw then put the frame together using brad nails and glue. The screen is folded over at the edges and stapled. The door hinge is a piece of leather and the door is held shut by a neodymium magnet that attracts the screw holding on the door knob. A light coat of danish oil finished it off. The entire project only took about 30 minutes.

In the image below, you can see the magnet countersunk into the frame. I drilled the hole to be snug and coated the inside of it with super glue before pressing in the magnet. I've done this before on several projects. The metal holding the knob on actually ended up working great and was more than enough for the magnet to attract. Here are the magnets I used:

There is a little green praying mantis in there. As I was putting in the last staples and testing the door in my garage this bug jumped in. I saw it all happen and just stood there in stunned silence. There aren't words I know of for coincidences this crazy. I could have searched my yard for hours and not found a bug this cool to show my son and here he is jumping over my shoulder and into our little bug cage just as I finish it. Perfect.

Here's a video of the enclosure in action. When my son woke up we studied and talked about the bug for a while and then went outside to try and let him go.

Saturday, June 21, 2014

Converting videos for Roku with Linux

I bought a Roku 3 and I wanted to watch some video files on it. Unfortunately it only supports a few video formats. Maybe this is a good thing as long as it does them well. Conversions are pretty easy using ffmpeg or avconv. An Internet search didn't turn up many results on how to do this, so here's a script I made to make my life easier after I figured out which codec works best.

avconv -i "$1" -strict experimental -c:v libx264 "`echo $1 | sed s/\.[^\.]*$//`.mp4"

To use it, put this in a file called and chmod +x
Then use it like this:
./ video_to_convert.avi

It will create an .mp4 from any formats that your avconv install supports. The .mp4 can be placed on a usb thumb drive and plugged into the side of your Roku 3. Find and play it using the Roku Media Player (download for free from the Roku channel store). Here's what it looks like:

Sunday, June 8, 2014

Forge update - gas adaption

When I woke up this morning I didn't know I was going to be converting my coal forge to gas. While on a family trip to Home Depot, I snatched my son up and we split away from my wife and daughter. I did my typical run around to see if anything good was on sale, look at esoteric stuff, and maybe find inspiration for a new weekend garage project.

I was never enthusiastic about playing with flammable gasses, but another project (you'll see in a future post) had me learning how to use Acetylene and Oxygen cutting tools. Acetylene and Oxygen is a potent mix fraught with all kinds of danger. After learning about and circumventing all of the ways to kill myself with it, I'm simply amazed more people don't get hurt using it. By comparison propane seems tame, so when I ran across some refractory cement at Home Depot it didn't seem like too much of a bad idea to build that gas powered forge I've been planning for a few years. Earlier in the week Harbor Freight had a sale on propane torches and I picked one up for next to nothing.

This propane forge is nothing more than a fire-proof container with a hole in the side where you can drive in fire. The pressure of the propane leaving the nozzle sucks air in via the Venturi effect. Here's a cool video by Mathias Wandell about the Venturi effect. Some forges have active air (like a supercharger) driving air by fan, but it's not really necessary. Especially for a back yard forge. The container keeps the heat in allowing you to approach the 3000 or so degree burning temp of propane in air over a wider area.

I created the fire-proof container by cutting/bending some steel mesh to fit inside of a paint can and packing the mesh with the refractory cement. I just used a pair of rubber gloves to smash it all in there and smooth it. Here's a picture of the process.
I cut a hole in the side and then very carefully welded in a piece of square pipe large enough to hold the torch tip. Another set of pictures showing how that works:

The front view of the forge after the first burn:

I scavanged a piece of 1/2" angle iron from a bed frame were throwing out. I cut that up and welded it together to create the stand which the paint can sits on. I sized it so that it fits on top of my coal forge, effectively using my brake disc coal forge as a heavy stable stand. Here's a picture of the whole setup sans propane. I took these pictures after I had already put away the propane for the day.

Before this forge will be truly usable, I'll have to figure out how to narrow the front opening. I'll probably use fire bricks, but not sure yet. I'm out of time this weekend.

Monday, May 5, 2014

12 days, Strata, and a fallen Marine

This is a post I wrote while on a plane in February. I'm just now getting around to posting it.


I spent last week at the Strata conference. It was a good time, saw some friends and soaked in some good presentations and discussions. James Burke gave a clever keynote on the last day. It was my favorite, here's a quote: "I'm an optimist, because to be a pessimist means to withdraw and not be involved. I like to be involved." I can relate to that, pessimism reeks of hopelessness and inaction. One of my previous Marine Corps commanders called me "disgustingly optimistic". 

Meanwhile, back in Virginia my very pregnant wife and her mother were digging out from under a foot of fresh snow. Despite that, I was optimistic about making it home on time and headed off to the airport early the next morning. At the airport the ticket attendant greeted me with a smile (she must be an optimist too ...or diligent service professional). The smile faded from her face when she looked down at her console. I knew before the words left her mouth that I wouldn't be getting home this day. Although every runway in the country was functional, the cancelled flights had a cascading effect and had knocked one of my connecting flights from the schedule. I cursed bad computer algorithms. For an hour I stared at the attendant while she made phone calls and typed on her computer, attempting to meddle with other people's schedules. She was able to determine that my choices were to spend another night in silicon valley or get to Denver and try my luck there. My disappointment in algorithms dissipated as I realized even the best flight scheduling system would fail with this amount of random intervention from empowered ticket attendants modifying it's solutions.

Denver is the home town of a fallen Marine I served with in Iraq. Walking through the airport, on the way to my flight, I was undecided if I should attempt to check in on his parents during my time in town. I wasn't sure how long I'd be there and it had been 5 years since I had visited them last. As though speaking directly to me the paging speakers in the airport kicked up and announced the name "Vandegrift". Apparently someone with this last name had missed a flight. They probably didn't hear the message. But I heard it; Matt Vandegrift was the fallen Marine I was thinking of and I was just knocked out of my fog of indecision. I was going to see the Vandegrifts. I sent his father an email from my smart phone in the jetway as I boarded my flight.

On the flight I popped open my trusty Dell XPS12. I had an interesting parsing problem on my work to-do list and some sample data to work with. After getting stuck and unable to access the Internet I ended up going through some of my old code looking for solutions I'd written previously. Unsuccessful, I started playing with one of my favorite pieces of code. twelve.c is 15 lines of c code that outputs all 64 lines of the 12 days of Christmas. I didn't write it, but I enjoy it; it's an old ioccc entry. I love pointing to this competition when people talk about static code analysis in the context of malicious developers. Good luck with that.

 #include <stdio.h>
 main(t,_,a) char *a; { return!0<t?t<3?main(-79,-13,a+main(-87,1-_,main(-86,0,a+1)+a)):
 main(2,_+1,"%s %d %d\n"):9:16:t<0?t<-72?main(_,t,
;#q#n+,/+k#;*+,/'r :'d*'3,}{w+K w'K:'+}e#';dq#'l \
q#'+d'K#!/+k#;q#'r}eKK#}w'r}eKK{nl]'/#;#q#n'){)#}w'){){nl]'/+#n';d}rw' i;#\
){nl]!/n{n#'; r{#w'r nc{nl]'/#{l,+'K {rw' iK{;[{nl]'/w#q#n'wk nw' \
iwk{KK{nl]!/w{%'l##w#' i; :{nl]'/*{q#'ld;r'}{nlwb!/*de}'c \
;;{nl'-{}rw]'/+,}##'*}#nc,',#nw]'/+kd'+e}+;#'rdq#w! nr'/ ') }+}{rl#'{n' ')# \
"!ek;dc i@bK'(q)-[w]*%n+r3#l,{}:\nuwloca-O;m .vpbks,fxntdCeghiry"),a+1);

The 12 days of Christmas is pretty repetitive prose (see what i did there?). Repetitive text lends itself well to compression tricks. The 64 lines of the song are generated by the above 15 lines of code, about 75% compression, I should easily be able to write something that could beat that if I were just going for compression and not obfuscation. Doing it without the aid of the Internet made it fun enough of a game to burn some time. Here's what I came up with.

echo -n $b64 | base64 -d | gunzip | cat

The original code has 840 characters in it, my in-flight solution has 495.
The version of 12 days of Christmas that they both print out has 2355 characters. It would have been nice to fit the whole thing into some code I could tweet, 140 characters. Not possible with direct methods, but that would have been interesting. There are a few minor optimizations I could have done, but nothing dramatic. This is probably close to as tight as it gets. If this is interesting to you, check out this great Dr Dobbs article about a long standing challenge to compress the random numbers generated by Rand in the 1940s.

To find out just how much of the text I could theoretically tweet, I wrote a small program to chart how many characters my approach uses for the first n lines of the text. The program just grabbed the first N lines and then implemented the method and output how many characters it ended up being. The red line shows how the 64 lines of the song eventually add up to the whole 2355 characters. The blue line is my algorithm.

The lines cross down in the left corner of that chart. That's where the benefit from compression overcomes the overhead of the surrounding code. Here's a close up of that crossover.

Unfortunately the crossover takes place when we're already past 140 characters. Meaning that any code you could tweet with this algorithm would be longer than just tweeting the plain text. Not cool. I was bummed when I saw this. It seemed like such a clever idea a few moments ago. Before I was able to come up with a different approach, the attendant told me I had to power down my laptop.

When I powered on my phone a message was waiting from the Vandegrifts. They'd love to see me and that I should let them know when to pick me up from the Airport. The ticket counter informed me that I wouldn't be flying out until the next day so I ended up spending the night and eating dinner with the Vandegrifts and talking about their son. We also spoke about survivor guilt. Matt's father gave me a copy of Viktor Frankl's book "Man's Search For Meaning". I read it on my flight from Denver to DC. It chronicled Viktor himself and his survival through a series of Nazi death camps.

Viktor writes that the "best of us" died in the concentration camps. Those not immediately killed by chance sacrificed themselves by standing up to the guards or "Capos" (privileged inmates). The worst kind of prisoners prospered and survived. That stuck out in my mind, because it truly shows some of the weight that those that survived made. They were often plagued by the gnawing suspicion that they may have survived because they were willing to compromise their ethics; even if it's not true. Life in a Nazi death camp was different than imprisonment by the Japanese like Zamperini endured as chronicled in the recent book Unboken. I finished Unbroken a few weeks after it was released and highly recommend it. Both stories, Unbroken and Man's Search for Meaning are very moving tales.

In Unbroken, Zamperini discusses his life after freedom. It's best summarized in his letter to his most cruel tormentor while in prison camp. Here's a video of Zamprini reading it. And the text:

"To Mutsuhiro Watanabe,

As a result of my prisoner war experience under your unwarranted and unreasonable punishment, my post-war life became a nightmare. It was not so much due to the pain and suffering as it was the tension of stress and humiliation that caused me to hate with a vengeance.

Under your discipline, my rights, not only as a prisoner of war but also as a human being, were stripped from me. It was a struggle to maintain enough dignity and hope to live until the war's end.

The post-war nightmares caused my life to crumble, but thanks to a confrontation with God through the evangelist Billy Graham, I committed my life to Christ. Love has replaced the hate I had for you. Christ said, "Forgive your enemies and pray for them."

As you probably know, I returned to Japan in 1952 and was graciously allowed to address all the Japanese war criminals at Sugamo Prison… I asked then about you, and was told that you probably had committed Hara Kiri, which I was sad to hear. At that moment, like the others, I also forgave you and now would hope that you would also become a Christian.

Louis Zamperini"

Viktor takes on a similar sentiment. He wrote in "Man's Search for Meaning" that "first time in my life I saw the truth as it is set into song by so many poets, proclaimed as the final wisdom by so many thinkers. The truth – that love is the ultimate and the highest goal to which man can aspire. Then I grasped the meaning of the greatest secret that human poetry and human thought and belief have to impart: The salvation of man is through love and in love. I understood how a man who has nothing left in this world still may know bliss"

Those that have ever been present for the death of another, especially from trauma, are destined to relive the events. To wonder "what if" a million times over. What if we wrapped the bandages a different way. What if we missed something that could have cued us to what was about to happen. What if we took a different road. What if.

I don't have a good answer for any of the questions or the weight that comes after. But I do believe both Zamperini and Viktor's messages. Regardless of what you do, where you've been, what somebody else has done to you, or what you've done; the true measure of worth and what you accomplish is ultimately measured by how you treat and your attitude toward others. That's the most important thing you can work on and that's something you can do right now. Don't live in the past, and don't live in the future. You're called to action right now.

February passed, as did April 21st (the anniversary of Matt's death) before I got around to posting this. There are probably only a handful of people that can appreciate a data compression discussion and non-technical book reviews in the same article. But, hey, this is my blog and I can post things in whatever combination I feel like, this isn't a popularity contest. Stranger things have happened on the Internet. If 5 other people can appreciate this, then that's great. Pretty soon I'll get around to posting something else about welding and really narrow down to my key demography which is apparently... well I have no idea.  

Here are some articles about Matt.
The first video posted to my youtube channel was a compilation of pictures and videos created by a Marine Gunnery Sergeant that I worked with. You'll see pictures of Matt and the rest of the team that I spent a year in Iraq with. Link here:

Friday, April 25, 2014

Counting unique file extensions recursively

Here's a quick example of the Linux command line tools working together to do something more complex.

I wonder what types of files are in this directory structure and and how many of each type there are?

A simple way to approach that would be to count the unique file extensions.

Let's start with getting a list of files.
eric@eric-Precision-M6500:~/workspace/temp$ find . -type f  

Ok... that worked. We can count them.
eric@eric-Precision-M6500:~/workspace/temp$ find . -type f | wc -l

But we're interested in just the file extensions, so let's get those. This regex matches a period followed by any number of non-period non-slash characters until the end of line and prints whatever is in the parentheses.
eric@eric-Precision-M6500:~/workspace/temp$ find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/'

We can do the same thing with sed (thanks Chris!).
eric@glamdring:~/Pictures$ find . -type f | sed -e 's/^.*\.//'

We want to count the unique extensions, but the uniq command only works when unique objects are next to each other in the input stream so we have to sort the extensions first.
eric@eric-Precision-M6500:~/workspace/temp$ find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort | uniq -c 
      1 html
      1 md
      2 png
      9 sample
      1 sh
      2 txt

Let's put some commas in the output so we can import into calc as csv and make a pie chart.
eric@eric-Precision-M6500:~/workspace/temp$ find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort | uniq -c | awk '{print $2 "," $1}'

That looks good. We'll redirect the output to a file and count unique extensions in a more interesting directory, all of the DARPA XDATA open source code on this computer.
eric@eric-Precision-M6500:~/workspace/xdata/code$ find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort | uniq -c | awk '{print $2 "," $1}' > extensions.csv

Extensions are a naive way to determine file type. The 'file' command is a little smarter, it combines several tests (filesystem, magic, and language) to determine file type. Here's an example of how to use it for our experiment.

eric@glamdring:~/workspace/randomproject$ find . -type f | awk 'system("file " $1)' | sed -e 's/^.*\://' | sort | uniq -c
    282  ASCII text
     18  ASCII text, with very long lines
      6  Bourne-Again shell script, ASCII text executable
     14  data
      2  empty
     45  GIF image data, version 89a, 16 x 10
      1  Git index, version 2, 606 entries
      1  Git index, version 2, 8 entries
      1  HTML document, ASCII text
     15  HTML document, Non-ISO extended-ASCII text

Here are some iterative improvements.

An awk-free version using the -exec option of find. 
find . -type f -exec file {} \; | sed 's/.*: //' | sort | uniq -c

A sed-free version of the awk-free vesion using the non-verbose output from file:
find . -type f -exec file -b {} \; | sort | uniq -c

If you have an idea for how to make this shorter or more effective, send me a note on social media and I'll include it.

Vagrant boilerplate

Not so long ago integrating software meant doing configuration management at the virtual machine level as integration took place. Taking snapshots and reverting if there was trouble. What you're left with a ~4Gb clunky file that you have to push around and convert to your eventual deployment environment. This is inefficient and risky. There's all the cruft that tags along as the builds take place. There's the complexities of image conversion.  Some painful workarounds sometimes need to take place to move it between environments (VmWare, deleting network interfaces). Hostname problems (Oracle, thanks for that). Security implementation guidelines also change at a moments notice or in different environments. And what if you want to deploy it to bare metal after all?
Thankfully, some better tools are available. If you haven't figured out how to use Vagrant yet, go learn it right now. If you know what a vm is and can use the linux command line, it will probably take you all of 15 minutes to master it.

Here's the website:

Basically vagrant has one configuration file where you specify a baseline OS image (a .box file), some other parameters about how you want it to behave. I say basically because this file does kick off and point to other config files that deploy your software. To kick all of this off you type 'vagrant up'.

Log in to the resultant system ('vagrant ssh') then test, develop, capture the changes you care about in the deployment files. When you mess it up, instead of reverting, wipe the whole thing out of existence with another command: 'vagrant destroy'. Lather, rinse, repeat. Within a short amount of time, what you're left with are some finely tuned deployment scripts that can take any baseline OS image and get it to where you want with one command. Vagrant allows you to spin up multiple vms at the same time so you can emulate and test more complex environments in this manner.

Although vagrant supports chef and puppet, my preference has been to use bash scripts to deploy software. Bash scripting is accessible to most people. When collaborating with groups of people from different organizations, it serves as common language. Recently, I've taken my boilerplate vagrant configurations and put them online.

It's best to separate parts of the deployment process. Don't write the commands that secure the system in the same file as the commands that deploy software components or data. Abstract it all. Then, when your deployment environment changes you only have to modify or switch out that one file. You can capture the security requirements for Site A and keep them separate from Site B. Want to deploy to a Site C? Build it out and you're only one command away from testing if everything works. If a security auditor asks how you configured your system, then send them the deployment file. If they have a critique (and they're good) they can make the recommended changes and send them back to you where you can just test them by running 'vagrant up' again.

My boilerplate includes simple snippets for how to create users, push around files, append to files, wget things to where  you want them, and other useful things that people forget when doing things from scratch (like writing the .gitignore). It should be easy to go through and modify it to do what you want it to do.

Here's a link:

It has Ubuntu as the base OS, here's a list of a bunch of other base OS's you can use without having to roll your own. Just modify the vagrant file.

Linux system administration

It's done just like this.

Monday, April 21, 2014

Weed superior firepower

The weather is warm and my in-laws are in town and spending time with the kids. This gave me a little time to take care of chores like de-weeding my front yard. The mulched area around the shrubs in front of my house has been invaded by weeds this spring.

Two minutes into picking them out with a small shovel on my hands and knees I got up and walked into the garage to design and build a better tool.'s funny that this is exactly how software engineering and data-research works. If you have a boring or repetitive task you should find or design and build a tool to make it easier. "Necessity" isn't the real mother of invention; it's laziness. Or, more aptly described, it's a natural outgrowth of intellectual curiosity. But I digress. We're talking about the scrap metal I melted together in my garage so I could dig in my yard, not metapsychology.

Here are the parts I used, and the end result. On the left is a 1.5" steel square tube, some 3/16" angle iron, and a short piece of reinforcing bar (rebar). The handle was cut from a piece of pine 2x4. The unstained handle was a quick prototype. I did a test-fit and then finished the welds while the head was on this handle (hence the burn marks and destruction on it). It caught fire several times while I was welding and grinding. I shouldn't have enjoyed that as much as I did. Every time I see something burning I think about this amazing Feynman description of fire. "The light and the heat coming out, that's the light and heat of the sun that went in!"

This square handle was easy to make and fit together. Everything was cut just using the table-saw fence and finishing it off with a chisel to get to the areas that the circular blade couldn't reach. Slice it down the center and slam home a wedge and this ended up being a solid way to put everything together. I think it's way better more classy than using bolts to attach the handle. Retro. 50,000 years retro.

I wanted something small that I could stomp on with all of my body weight to sever or dig out stubborn roots if I had to. Here's a short video of how it works. I filmed this with my phone in one hand and the tool in the other; with the benefit of both hands the tool is really efficient.

Here's me lining up on a weed:
After a successful extraction, roots and all:

The welding on the tool is a little excessive. Part of the reason for undertaking this project was to teach my father-in-law how to weld, so we kept taking passes on the metal. It's interesting to see how others interpret verbal instruction or how they pick up new tools and methods. His initial pass he waved it back and forth like a paint spray gun; I didn't expect him to do that. On the second try, I made sure to explain that you have to hold in the same spot until the metal starts to melt and then slowly lay a bead of weld. I didn't bother grinding things down very much afterward, just spray painted it and hung it on the handle. That's the fun part of building garden tools, they don't have to be pretty.

A note about the steel. This is mild steel, and that's ok. I had some high carbon steel that would have been harder, but I'd prefer a digging tool to turn an edge rather than chip. I know this tool will be slammed onto rocks and concrete, but any damage done can easily be pounded (preferred method for those of us that own good anvils and hammers) or filed out (last resort) before resharpening with a file.

This build was inspired by Wranglerstar. If you like this kind of stuff check out his videos.

Monday, March 10, 2014

pull up bar build

A friend created a wooden pull up bar with a design similar to this and I've been wanting to build my own version for a while. I was never satisfied with any of the wooden designs I could come up with. They seemed either too bulky or too weak. Nothing I'd want hanging on the wall in my small home office.

This Saturday I was out playing around in the garage and looking for an excuse to weld something. This is what I came up with just sticking some scrap metal together. I think it ended up being perfect for what I wanted and it came together fast. The longest part of the build seemed like it waiting for the paint to dry so I could try it out.

Safety notes:

It's March here in Virginia and I'm the only person at work with a tan on my forearms thanks to not wearing any protection while welding this. I'm sure the youtube safety patrol will be all over this. My left arm is a little burnt from the welding flash.  It's only enough to be uncomfortable for a day or so, like a light sunburn. No big deal, but next time I'll put on my welding jacket. I have flashed my eyes before. If you've ever wondered about how painful this is, welding flash to the eyes feels like having sandpaper under your eyelids for a few days. So not fun. It's the exact opposite of fun actually. I'd advise not messing around with welding eye flash. Spend the money on an auto-darkening helmet so you're not messing around with your helmet every time you position the arc. It's been probably ten years since I've messed up on that one and just thinking about the welding sunburn on my arm has me re-living how obnoxious and painful eye flash was.

In some of the drilling scenes you'll hear my son in the garage "helping". He was far across the garage playing with some wooden blocks and wearing his safety glasses. I scoot him off to his mother whenever I weld, grind, or plug in the dangerous saws.

Thursday, March 6, 2014

End Grain Cutting Board

My daughter was born last Tuesday. I've been spending a lot of time around the house helping out my wife and caring for the new little girl. I've somehow managed to get an extraordinary amount of work done over the last few weeks. Those of you that have gone through the modern process of having a child (this was my second) know that there's a lot of waiting around in the hospital. As a dad, this means watching mom and the new baby sleep. Then, when you do finally get home it's 30 minutes of chaos every few hours during feeding time, then more down time. I've re-factored the entire code base that generates the DARPA Open Catalog.

I even managed to sneak out into the garage for a few hours, clean it up and dive into a project. I ended up making two end grain cutting boards. One I sent home with my parents who were in town helping out with the new baby. The other is now in my kitchen. They were probably 4 or 5 hours total work, 3 of which was sanding and finishing.

This board is made out of 8/4 Maple and 8/4 Walnut with some Osage Orange left over from a bow project last year (archery bow). These are very close grained woods and lend themselves to this type of project. I had some purple heart and ebony laying around which would have worked great too but I decided not to sacrifice them for this project.

I didn't get the whole build process on film, just the last half. But, there are many videos about how to cut and glue these boards together.  The Wood Whisperer probably has the best one: If you want to build your own, use his plans and tips. I didn't significantly change anything here except for the addition of some rubber feet. I put them on before I applied the mineral oil and wax; they seem to be sticking well. I did this because in a follow up video Marc Spagnuolo shows how his board cracked after laying in some standing water on the counter top.

Update 1: The board was glass smooth until the first time I wiped it off with a wet rag which raised the grain slightly. I suspected this was because I didn't apply enough mineral oil and wax the first time around. On my second application I applied a liberal amount of butcher block conditioner on the board (wax and mineral oil mix) and then swept over the board with a heat gun until as much of it was soaked up as I could get the board to take. It seems to be fully conditioned now, but I've read that I may have to do this several times.

Update 2: The sticky feet were a bad idea. I found some rubber feet on Amazon that shipped with prime. I pre-drilled some holes and put these on. They've been working perfectly for months now.

Update 3: With the lessons learned from this project, I later took my pieces of purpleheart and made another cutting board for my old neighbors. Here's a picture of that finished project.