The graph below from Tomaž Kovačič‘s study shows only a small amount of the data he collected in his analysis. If you are curious of how he compared these algorithms, I highly recommend you head over to his post. He does a great job exposing the details behind his analysis.

Goose's standing among other algorithms tested

So what is Goose used for at Gravity and why have we open sourced it?

Goose’s wiki provides a very detailed explanation about what Goose is and how it works, and also touches on the original need we had at Gravity behind its creation. Jim Plush wrote the first version from the ground up on his own and only recently gave me commit access to the repository. By the time I got into the project, it had all the bells and whistles required to compete in the analysis completed by Kovačič. My contributions to Goose have been to extend it to allow for more specific extractions of additional meta data outside of the primary content and have no effect on its standings above.

Such a utility can be applied to a wide variety of web content analysis problems, and I’m really glad Plush decided to share it with the rest of the open source community. At Gravity, we have been building a lot of exciting (to me at least) technology and most of it is held dearly by us and needs to remain a company secret as they make up a large part of our company’s overall value. When it comes to analyzing the content out here on the web, Goose can be looked at as our trusty messenger delivering our system plenty of content to analyze without a lot of the noise that comes along with it on the web pages the content is sourced from.

If you are looking to mine some of the golden nuggets of information that is buried under a ton of ads, peripheral links, site menu structures, and other distracting noise, then why not take a look at what Goose has to offer? If you find anything you think Goose may be lacking or have some ideas on anything else that may be improved, let us know on our Github repository: https://github.com/jiminoc/goose

Well things seemed fairly normal in the conversations I was having except it did seem that she was having a harder time than usual understanding what I was saying. I had a long wait between buses and I was getting pretty hungry, so I started instant messaging questions about my dinner options for home. In come responses from my wife that they had “GoodStuff” for dinner and when I asked if there was any left for me, a resounding “Yes!” with smileys came back. I knew that they must have finished eating hours ago, so I made a request for her to start reheating it so that I could eat quickly and then move on to doing bedtime for our two daughters. I was so happy when a quick IM response came back saying: “Sure! OK!” and again a long line of various smileys.

More small chat continued until I arrived home to find an empty table and empty stove. Although it was nice to see that everything was so neat and clean, I was a little disappointed that there was no hot dinner for me after the IM conversation we just had. I then noticed that my wife was busy with our laundry and my 7 year old daughter (aka: Thing Two) was next to her holding my wife’s phone (we use our phones for instant messaging). Not only was Thing Two happy to see me, she was also laughing a lot more than usual. I asked my wife about the IM conversation we had moments ago, and she looked a little confused. This is when Thing Two jumps up and says: “I fooled you Daddy! You thought I was Mommy!” We all had quite a laugh.

It was just shocking at how I was not able to notice the difference. My wife tends to be very terse in her IM communication, so it did not seem odd for me to ask a long question and then receive a small “ok” response. Boy has Thing Two come a long way in her pranks. I’m both proud of her and a little scared for what we’re in for as well.

LOL O_o

the plight of my klout score

I tend to focus less on my social networks when I have my head down coding. This time however, its effect on my klout score is pretty dramatic. LOL

I guess this is just a consequence of the typical life:work balance, but I wonder how other tech professionals maintain such a high score while also paving new roads in their field.

A couple of examples of the type of tech peeps I’m talking about:

1. Jeff Atwood – 73: klout.com/codinghorror
2. Matt Cutts – 73: klout.com/mattcutts

Anyway, I’m not complaining here people. I am just confused about how others manage to keep up their tweeting/blogging while “deep in the cut” of some tech project.

Do any of you have any tips? Please speak up here and let me know.

Image credit: audi_insperation

Things have been so exciting working @gravity this year and I am amazed at how much we have accomplished in the last few months. All of this excitement and hard work has worn my not-twenty-something body out just a tad and my family has become quite used to my MacBook on my lap during weekend afternoons.

So…. How nice it is that we all get a 3-day weekend to relax and spend some quality time with those people (and/or cats) in our lives that do not work at Gravity? I have also been able to work on this blog once again. After a nice day out with the family on this beautiful spring day, I found myself curious about what I needed to do next… NOTHING!

How great is that? I was really dumbfounded for a while until my brain chimed in with a wonderful suggestion: Relax! Well that is just what I am doing right now.

Here’s to having a moment to relax and accepting it! Hope you get a moment or to yourself.

peace.

1,317 of 4,947 songs added... OH HELL YEAH!

I just gained access to Google’s Music Beta and for the first time, I think my personal music library may be smaller than the amout of cloud storage available for free!

It is truly amazing just how far we have come from the the early days of cramming mp3′s into a JPEG image to store on a free image hosting site back in the 90′s. Napster not only broadened what was possible for music online, but also inadvertently set us all back a decade of fighting to truly OWN the music we legally purchase from the big music labels.

Yes, I know there are a lot of you that have second thoughts about giving so much to the Google Collective, and I have no beef with you and your own convictions.

I for one welcome our new online music overlords!

Sometimes you just can’t find a single pair out of the entire bunch.

The actual SQL code for extracting all of my blog posts looks surprisingly simple:

SELECT  [Subject], PostDate, FormattedBody, 
          dbo.old_url(PostDate, [Subject]) AS old_url, 
          dbo.make_slug([Subject]) AS slug
FROM  dbo.cs_Posts
WHERE   UserID = 2102 
  AND   SectionID = 4 
  AND   PostLevel = 1 
  AND   IsApproved = 1 
  AND   IsLocked = 0

But if you look closely at it, you’ll see that there are to scalar functions in there: ‘dbo.old_url‘ & ‘dbo.make_slug‘. I was surprised to find not find any slugs in the CS DB tables. I assume that all of that logic is being handled from the compiled ASP.NET application itself because there was nothing in the tables, stored procedures, or even functions that did anything related to calculating/parsing URL slugs from post titles. To make matters worse, since my site was not in a running state (due to hosting shenanigans), I had basically just my memory along with the 404 logs on the new WordPress site to help me reverse engineer the rules for converting titles to slugs. This is best represented in my ‘dbo.make_slug’ snippet below:

CREATE FUNCTION [dbo].[make_slug]
(
	@post_title nvarchar(256)
)
RETURNS nvarchar(500)
AS
BEGIN
	-- Declare the return variable here
	DECLARE @slug nvarchar(500)
	DECLARE @clean_title nvarchar(500)


	SET @clean_title = LOWER(dbo.deDupeSpaces(dbo.removePunctuation(@post_title)))
	SET @slug = REPLACE(@clean_title, ' ', '-')

	RETURN @slug

END

And that is used by ‘dbo.old_url‘ here:

CREATE FUNCTION [dbo].[old_url]
(
	@post_date datetime,
	@post_title nvarchar(256)
)
RETURNS nvarchar(500)
AS
BEGIN
	-- Declare the return variable here
	DECLARE @url nvarchar(500)
	DECLARE @y_m_d nvarchar(10)
	DECLARE @clean_title nvarchar(500)

	SET @y_m_d = CONVERT(nvarchar, @post_date, 111)
	SET @url = '/archive/' + @y_m_d + '/' +  + dbo.make_slug(@post_title) + '.aspx'

	RETURN @url

END

There are still two more functions remaining (if you have been paying attention) that are used by ‘dbo.make_slug‘ and that is where the real fun comes in. First of these is the simpler ‘dbo.deDupeSpaces‘ which cuts all repeating space characters down to a single space:

CREATE FUNCTION [dbo].[deDupeSpaces] 
(
	@input nvarchar(500)
)
RETURNS nvarchar(500)
AS
BEGIN
    /**
    *  Based on Nigel Rivett's SQL script found: 
    *    http://www.nigelrivett.net/SQLTsql/RemoveNonNumericCharacters.html 
    */
	DECLARE @i int

	set @i = patindex('%[ ][ ]%', @input)
	while @i > 0
	begin
		set @input = replace(@input, '  ', ' ')
		set @i = patindex('%[ ][ ]%', @input)
	end

	RETURN @input

END

And the more impressive and pretty much identical to the script I found originally written by Nigel Rivett:

CREATE FUNCTION [dbo].[removePunctuation] 
(
	@input nvarchar(500)
)
RETURNS nvarchar(500)
AS
BEGIN
	/**
	 *  Based on Nigel Rivett's SQL script found: 
	 *    http://www.nigelrivett.net/SQLTsql/RemoveNonNumericCharacters.html 
	 */
	DECLARE @i int

	set @i = patindex('%[^a-zA-Z0-9 ]%', @input)
	while @i > 0
	begin
		set @input = replace(@input, substring(@input, @i, 1), '')
		set @i = patindex('%[^a-zA-Z0-9 ]%', @input)
	end

	-- Return the result of the function
	RETURN @input

END

So all of this so far is just to get my posts out of the CS DB in a format close enough to what I’ll need to stuff into my WordPress DB. In order to continue, I just ran the simple query (snippet at the top) and exported the results to an XML file. Now I could finally shutdown the virtual instance of Windows 7 that was eating up my MacBook’s resources and burning my lap from the CPU pegging.

The rest is pretty straight forward. I was unable to find any WordPress Plugins so to assist me in this completely custom hackery, so I thought a brute force insert directly into my WordPress mySQL DB was a great idea. I first imported the XML file into a new table that I called cs_posts. This table’s structure is identical to the original query used to export it. Once this was done, I built a basic INSERT INTO …  SELECT query to import these CS posts directly into my WordPress posts table:

INSERT INTO wp_xxxxx_posts 
	(post_author, 
	post_date, 
	post_date_gmt, 
	post_content, 
	post_title, 
	post_status, 
	post_name, 
	post_modified, 
	post_modified_gmt, 
	guid) 
SELECT 2 AS post_author, 
	cs_posts.PostDate AS post_date, 
	cs_posts.PostDate AS post_date_gmt, 
	cs_posts.FormattedBody AS post_content, 
	cs_posts.Subject AS post_title, 
	'draft' AS post_status, 
	cs_posts.slug AS post_name, 
	cs_posts.PostDate AS post_modified, 
	cs_posts.PostDate AS post_modified_gmt, 
	cs_posts.old_url AS guid
FROM cs_posts

From this point, all that was required was for me to correct any permalinks that did not match up to the slug I had calculated. But I also wanted to get 301 redirects in place for all incoming requests looking for /archive/YYYY/MM/DD/some-post-title-slug.aspx to find their way to the new URL /YYYY/MM/some-post-title-slug. This was much easier than I anticipated due to the luxury of John Godley‘s Redirection plugin. This gem of a plugin makes my introduction to the WordPress ecosystem a dream come true. In fact, after I set it up on both this site and my root: robnrob.com site, I was able to populate the redirection item table his plugin uses to skip the need to enter in each post’s specific redirection. The plugin also has an option for regex-ish pattern matching, but a lot of the permalinks I ended up with on WordPress would not directly transpose from the basic:

url pattern: /archive/(d+)/(d+)/(d+)/([a-zA-Z0-9_-]+).aspx
redirect to: http://robbie.robnrob.com/$1/$2/$4

In the end, I lost out on previous comments, categories, and tags, but what I gained was a much more reliable hosting environment and a much more enjoyable platform to hack on. Also, to be honest, I had only a handful of comments anyway.

Google Top Search Result for Robbie Coleman

Amazingly, just after one day of me installing/configuring this new WordPress blog, a search on Google for my name: Robbie Coleman returns this site as the very first result!