code

i've done a lot of netbooting-related stuff at work. a *lot*. i could probably talk your ear off about PXE, WDS, and a million other bits of netboot-related jargon in my sleep.

however, there was one quirk with Ubuntu that was driving me crazy. Ubuntu Server 10.10 and before worked flawlessly. however, starting with Natty Narwhal, i'd do a netboot install that would proceed as normal...until i booted into the system. at that point, i couldn't actually see anything on the screen. it was clear the computer was booting: it would go through the BIOS, and only stop displaying video when it went into the operating system. i was stumped.

this happened on any machine i tried the netboot install on: Dell, Supermicro, HP, no-name, you name it. it happened on new machines and old machines. it always happened. i googled and googled with no luck, and finally just forsook netboot installs of Natty in favour of burning discs.

Oneiric Ocelot came out earlier this month, and i was hoping this problem would be fixed. i put it up on the PXE server, and no dice. i was annoyed, but i was determined. i cast a wider net in my search for a solution, looking at more general forum threads about Ubuntu video issues, since limiting it to just netboot install advice was getting me nowhere.

finally, i figured out the problem, and how to get around it. the problem was that trying to display the splash screen was completely borking the virtual terminal. (i figured this out because, when i hit Ctrl-Alt-F2 on one of the "hosed" installs, that virtual terminal came up just fine.) the problem can be avoided by preseeding the Ubuntu install to call a post-install script that removes the splash screen instruction from the default boot line generated by GRUB2.

to implement this, put the following lines at the end your preseed file. (or, create a preseed file with these lines, if you're not using preseeding with your Ubuntu install):


# this is just a post-install script.
#
# update Grub to get rid of that splash screen that borks the
# video on netboot installs of 11.04 and later
d-i preseed/late_command string \
cd /target; \
wget http://bigtruck.minazo.net/unbork_splash.sh; \
chmod +x ./unbork_splash.sh; \
chroot ./ ./unbork_splash.sh; \
rm -f ./unbork_splash.sh


this pulls a script down that fixes the GRUB issue, executes it in a chroot of the new install, and then deletes the script.

for the sake of openness, here's the script it pulls:


#!/bin/bash
# by rogueclown, 2011
# WTFPL (Do What The Fuck You Want To Public License)
cd /etc/default
sed "s/quiet splash/quiet/g" grub > grub.new
mv grub grub.orig
mv grub.new grub
update-grub
exit 0


voila: the box reboots, and you've got video.

why a server, much less a Linux server needs a splash screen, i'll never know. i love Ubuntu Server, but i'm quite angry that such a useless feature as a splash screen causes such an annoying problem as borking the virtual terminal on which it tries to display.

i've been batting around in my head just blogging about the random bits of tech knowledge i learn along the way, or computer-related things i do. that way, it'll be a way to commit that stuff to memory for my future reference, as well as document the trajectory of things i'm doing and learning.

in the spirit of that last entry, i'll start with a little bit of code i wrote earlier tonight. i work in a data center, and my duties are as varied as the calamities and successes that befall our customers. however, one of the fairly common tasks i have to perform is preliminary investigation of abuse complaints: figuring out which customer sent the complained-about spam (much less, if it was a customer at all...), notifying the customer of the complaint, and escalating it if it's particularly messy. basically, this involves a lot of whois queries (or dig queries followed by whois queries, for those spam complaints that only provide domain names and not IP addresses).

it was a particularly long night of spam complaint lookups tonight, and i started to get very annoyed about having to type "whois" or "whodig" (a little one-liner i wrote a few months ago to automate the dig-then-whois process) before every single IP or domain i cut and pasted from the spam headers. i wished i could just paste the IP or domain at the command prompt, hit enter, and have the shell read my mind that i wanted a whois lookup.

so, as any self respecting lazy IT geek would do, i wrote a script that does just that.

#!/bin/bash

# whoprompt.sh
#
# an interactive whois command line
#
# this will run whois if an IP is typed at the prompt
# and whodig (a dig-and-then-whois script) in response to a domain name
#
# nicolle neulist
# December 25, 2010
#
# * ----------------------------------------------------------------------------
# * "THE BEER-WARE LICENSE" (Revision 42):
# *  wrote this file. As long as you retain this 
# * notice you can do whatever you want with this stuff. If we meet some day
# * and you think this stuff is worth it, you can buy me a beer in return.
# * ----------------------------------------------------------------------------

# define constants
SHELLPROMPT="whois> "

# define function to test for a valid IP address

function valid_ip()
{
	local  ip=$1
	local  stat=1

	if [[ $ip =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
		OIFS=$IFS
		IFS='.'
		ip=($ip)
		IFS=$OIFS
		[[ ${ip[0]} -le 255 && ${ip[1]} -le 255 && ${ip[2]} -le 255 && ${ip[3]} -le 255 ]]
		stat=$?
		fi
	return $stat
}

# define function to run a dig query on a domain name and then a 
# whois lookup on the corresponding IP

function whodig()
{
	whois $(dig $1 | grep -A1 "ANSWER SECTION" | grep "IN" | awk '{print $5}')
}

# main loop
printf "type \"help\" for help.\n"
printf "type \"exit\" to exit.\n"
while [ 1==1 ]; do
	read -p "$SHELLPROMPT" -e address
	if [ $address == "help" ] ; then	# print help text
		printf "\ntype an IP address to run a whois lookup.\n"
		printf "for any other text, it will attempt to run a\n"
		printf "dig query, and then a whois lookup on that.\n\n"
		printf "it will return the same success or error text\n"
		printf "as a normal whois or dig lookup on linux.\n\n"
		printf "to end the program, type \"exit\".\n\n"
	elif [ $address == "exit" ] ; then 	# "exit" quits the program.
		exit 0
	else							# whois/whodig routine
		if valid_ip $address; then	# test IP
			whois $address		# whois on valid IP
			printf "\n"
		else
			whodig $address		# assume domain if not IP
			printf "\n"
		fi
	fi
done

Hopefully you'll get some use out of this little bit of code, especially if you're also in a position that requires lots of abuse complaint investigation, or anything else that requires a lot of whois lookups.

I have made a few much-needed site updates today!

First of all, I have made a new page about Arduino music, and linked it off of the projects page. It contains links to the code for my Arduino musical projects, as well as links to various pages that I have found helpful in exploring the subject.

Secondly, I have updated my presentations page. I added the slides for the talk about instilling a more welcoming culture in the hacker community that I gave at Notacon 7 this month, as well as the slides from the FDCC Virtual Machine talks that I gave at Pumping Station: One's Pecha Kucha Night and at Neighborcon NYC in December. Also, I added the slides for my upcoming talk about Arduino music that I am giving at Penguicon on Saturday, May 1.

If you're going to be at Penguicon this weekend, please come to my talks! I am giving a talk entitled From Microcontroller to Maestro: Music on the Arduino at 3:00pm on Saturday, May 1 in Private Dining Room I. At 4:00pm that same day, in the same room, I will be talking on a panel entitled What the Hack? Hackerspaces in Detroit and Beyond with Russ Wolfe and Nick Britsky of i3Detroit, as well as Mitch Altman of Noisebridge.

Recently, the Chicago Transit Authority released its CTA Bus Tracker API. For those of you who don't live in Chicago, CTA Bus Tracker is just about the greatest thing ever. It allows you to look up online, in real time, when the next bus is coming to a stop. It was rolled out over the last few years, and within the last year or so, has been available for every bus. My only major complaint about it was that it was only available via the Bus Tracker website, so if you did not have a smartphone with a web browser, you were out of luck. But, there was very little I could do, since there wasn't an open API for accessing their Bus Tracker data.

Yesterday, I saw a few tweets about various Canadian public transit systems that had opened their APIs to varying degrees. Curious, I looked up whether CTA had done anything in the last few months to open up the Bus Tracker API. They had! Back in September, they posted it online. They require you to make an account on Bus Tracker and apply for a developer key, so I decided to go for it and see if they'd allow me to start using it.

...and, I got my developer key this afternoon!

This led to me spending the vast majority of the afternoon and early evening playing around. The Bus Tracker API itself is pretty straightforward, but I hadn't yet learned the python commands for creating DOM objects and parsing XML. (In fact, I didn't even know what a DOM object was until this afternoon--I just assumed that there had to be an easier way to parse XML than parsing it with regular expressions or pattern matching within the strings.) At this point, I have gotten the hang of pulling the Bus Tracker XML data and parsing it, which means that I can start to do something useful.

My thought is to do a Twitter interface. I enjoy working with the Twitter API, and back when I didn't have a smartphone (but could use Twitter by SMS), I wished there was a way I could text for bus times. A good Twitter interface for Bus Tracker would solve that problem.

Of course, the challenge is going to be crafting a "good" interface. The code is not going to be the difficult part. Finding a way to make a Twitter interface that is intuitive to use is going to be the difficult part, since there is so much information that Bus Tracker needs to give the user the times: bus route, direction, and stop. The difficulty will come because people will format their queries in different ways (for example, referring to the same stop as either Belmont and Sacramento or Sacramento and Belmont or Belmont/Sacramento or Belmont & Sacramento, et cetera), and it would be extremely obnoxious for the user to have to be all that picky about their formatting or street name order if they are rushing to find out when their bus is going to come.

It seems like the hottest thing on the internet right now is a Google Wave invite. The day that Google began to offer Wave invites, I felt like I was the only person on my Twitter feed who was not tweeting either about the fact that she had one, or the fact that she wanted one. Since then, I still see some tweets about it, but it has mainly calmed down on my feed.

However, Twitter is still abuzz with people passing links to supposedly free Wave invites around. It seems simple enough to the unintiated: give some website your Twitter name and your email address, retweet the link to their website, and get your hands on one of thousands of invites. Sounds simple, right?

But...anyone can put up a site claiming that they have Wave invites, or anything else. It has all the marks of a scam: someone you don't know has harvested your email address, and they can send whatever spam they want to you. They've also matched your email address to your Twitter account...which makes it easier to crack your accounts, especially if the passwords to them are the same.

I was having a conversation about this with Hellekin on Twitter earlier today, and he suggested putting up a site that culled the tweets that passed around the spam links, and call the people out on the fact that it's probably a scam. I am not all that adept at putting up websites--however, I really do enjoy making Twitter bots, and started becoming very good friends with the Twitter Search API last week, while writing an IRC bot that (among other things) searches for tweets that reference my hackerspace. Thus, I took a little code from that IRC bot, took a little other code from the Kanye Bot, tweaked it a little bit, and made a bot that digs up tweets that are likely to be from people who have fallen for the scams. It lightheartedly tells them that their email address has now been taken by a scammer, and advises them that it may be good to change their password.

The bot is posting at @WavePwned. Right now, the way it's coded, it hits a few false positives, since the search terms are google wave invite http: So, in addition to hitting people who are tweeting links to span sites about Google Wave invites, it also hits people who are linking to articles about Google Wave invites. However, it is almost impossible to craft something more specific and hit such a large amount of the people propagating these phishing links, since there are so many new links made with the same kind of spam, and the phrasing of the spam tweets changes so often.

Hopefully, this little bot will alert at least a few people to what a hoax all of these Google Wave invite sites are, and make them think a little more before giving some random website their information. The morals of the story: if it sounds too good to be true it probably is, and think before you give out your social networking or email address information.

(thanks, @hellekin, for the idea!)

Sometimes, coding is serious business.

And, sometimes, coding is not serious business at all.

I've done a little bit of playing around with Twitter in the past. Last year, I wrote a simple twitter client bash script, so I could update my status from the command line. I've gotten some work done on a twitter bot that actually tries to fool people into thinking it is human. I haven't put it online yet, since I need to finish writing the content that implies that this bot is up to interesting things during different parts of different days.

However, this morning, Rob T Firefly gave me the idea to take that silly Kanye West internet meme and turn it into a Twitter bot. Right now it polls the public timeline every thirty seconds, and searches for tweets that say "thanks" or "thank you." If it finds any such tweets, it responds to the writer of the tweet and tells them that it is happy for them and will let them finish, but that whoever tweeted just before them in the public timeline had one of the best tweets of all time. I am also adding functionality that spits back a random Kanye West quote from a list if anyone @ replies to the bot.

The code is still a work in progress, but the latest version of it is up on my projects page. I'm quite pleased that it has been running on Twitter for about ten hours so far, and the account has not yet been deleted for being a spam bot.