Linear time, Logarithmic time, and Databases

Database tables are usually just arrays of data, sometimes contiguous in memory, and sometimes not. To scan a table for the right answer on an arbitrary field would be O(n) in the worst case, since database tables are not guaranteed to be ordered (it's implementation specific, by my understanding, meaning that so long as the queries function properly the DBMS can store the data however it likes).

When you have 20 records, no big deal. However, when you have 20,000,000 records, this quickly becomes an issue. To fix this seeking issue, there is the concept of an index. This creates a meta-table with additional information on the table that makes querying easier. This meta table contains the column of data being indexed, and the indexed column is sorted and put into a binary tree variation (the most common is a red-black tree, because it offers guarantees on the worst case run time). So when the database executes its query, it looks at the where clause to see if it has an index for that set of conditions. If it does, it searches in the index to get the direct pointer to the row in memory. By virtue of using a pseudo-balanced binary tree, this means the search only takes O(log n) time.

However, there are caveats here too. Database Management Systems (DBMS) are amazingly fast at some things, but do very little inference on their own. Let's take an example of a users table:

Users
------
id: integer
name: string


A common query we might use is Select * from Users where id=5. Without an index, this query needs to scan every entry in users. With an index on the "ID" column, this runs in logarithmic time. Same thing with Name - if you have an index, it's logarithmic, otherwise it's linear. Now let's look at a more complex query:

Select * from users where id < 5 AND name="matt";

This can't make use of just an index on the column ID, or an index on the column name, because even if it finds the ID, it will still need to scan the table for the name. So one thing we can do is create a multi-column index - one on each pair of <id, name> from the table. This removes the table scan, but introduces a weird quirk of DB behavior. Say we have an index on the column pair <id, name>. Now take the following query:

Select * from Users where name = 'matt' and id < 5

You would think this query could use the <id, name> index, but what actually happens is that because there isn't an exact match the DB does a table scan instead. The reason is because the SQL interpreter doesn't want to handle the parsing complexity necessary to allow for arbitrary interpretation of indices. In essence, the order of the columns in the index *must* match the order of the columns in the where clause in order for that index to be used

Something I found useful when creating database indices in Rails was the following code:

    [:column_1, :column_2, :column_3].permutation.each do |combo|
      # Allowing the automated names results in names that are too long
      add_index :table , combo, name: combo.join("_")
    end

.permutation generates all the permutations of the preceding array, making doing things like 3-column indices with all variants a snap

Note that this is only true for relational databases like SQL Server, Oracle, Postgres, MySQL, and so on. NoSQL databases use different data structures behind the scenes, and are more optimized for text-base and document-based searches. Additionally, some providers may improve on this in various ways to enhance DB performance, so while you can use the above as a general guide, it's usually a good idea to verify this behavior when picking a new database system.

Georgia Tech - Two More Weeks Down!

So last night I finished module 3 in my AI for Robotics course that I am taking as a part of Georgia Tech's Online MS in Computer Science. Things have somewhat settled down into a routine, with my various classmates working through the course at their own speed. Due to a complete lack of time in pretty much my entire life right now I'm simply trying to keep abreast of the weeks as they are due according to the schedule provided at the start of the term. Problem sets are due each Wednesday, and are submitted through Udacity's software. An instructor then goes through and ports these into something called T-Square, which I understand to be Georgia Tech's grading system. I could be wrong, though.

Assumed background knowledge

One of the things I'm noticing is that the course is starting to assume some background knowledge that not a lot of students may have. This is unavoidable in pretty much any course, as I have found out too often during my stints as a college professor. However, there are some ways this could be mitigated. For example, this is my first course at Georgia Tech, and my understanding is that it is targeted towards the end of the program. I took it because I found the subject interesting, but I could see where this might be causing me some trouble.

One example is that the course makes some decent use of techniques from Linear Algebra. I have a healthy grounding in this thanks to my time at DePaul and a few years spent in the pixel mines, but more often than not this type of math is an elective - at best - for computer science undergraduates. The problem is that there wasn't much in the course description that indicated that this type of math would be used. Statistical analysis is also featured heavily in the course, and while there are ample resources available for coming up to speed most students have no idea that they will need to do so before the course starts. I had around a month before classes began where I could have been preparing myself, but as it was I kind of walked into the course blind.

Interesting Problem Sets

One of the things I do find interesting is that the problem sets at the end of each lecture are actually not a significant portion of the grade. In fact, they're worth 3 points each (compared to a total upwards of 70 points for the two projects), and are graded on a pass-fail basis. To quote the professor, they essentially serve as a kind of "attendance check." That's all well and good, but if I am putting forth effort towards these assignments I'd like to see just a little bit more credit given for my efforts. Not much I can do about that, though, and I can definitely see the instructor's point.

One thing I do like about these problem sets is the use of automated grading. At first it's used in a fragile manner, along the lines of "Does the program output include this string," but the most recent assignment used statistical analysis in its grading to determine if the provided code was correct. It's pretty rewarding to see code that you wrote indicated as correct against an arbitrary data set with around 95% of accuracy.

Moving Forward

One thing I'm kind of leery of is wondering when the lack of prerequisite knowledge is going to bite me. I'm worried about the week 4 subject material, as it depends a lot upon dynamic programming and some aspects of machine learning - to the point where I'm reading chapters from a ML textbook in order to increase my knowledge base. It leads to one of the things I've been most concerned about, which is adequately scheduling time to devote to my courses.

So far the coursework takes me between 2-4 hours a week - a surprisingly low amount. However, I know that this is not the case for other courses, such as Machine Learning which can demand upwards of 40 hours of additional time each week, and in many cases I'm waiting for the other shoe to drop. I suppose at this point all I can do is wait and see, and hope it doesn't get too bad.

I'll try to provide these posts more frequently but, as I mentioned above, life happens :-/

More posts at Elite Gaming Computers!

This week I had a nice contrast in my articles. On the one hand I covered the ASUS Z97-WS, an absolute beast of a motherboard. On the other hand, I covered the Raspberry Pi for emulation and HTPC usage. The contrast pegged my chuckle meter, at least, as I found it interesting to go from "God of POWER!" to "Small but mighty!".

Anyway, check out the articles here:

http://elitegamingcomputers.com/raspberry-pi-htpc-emulator-on-a-budget/
http://elitegamingcomputers.com/the-asus-z97-ws-a-beast-of-a-board/

Georgia Tech OMSCS - Week 1 Recap

I know, it's only Thursday!

So I just finished my first "week" of class for Georgia Tech, and I wanted to get some of my thoughts down on virtual paper. I'd like to chronicle my adventure semi-frequently in this manner, so please bear with me if this isn't your cup of tea.

Some Background

So one of the things I'm willing to admit right off the bat is that I was pretty nervous. Given my prior trials and tribulations trying to even get into the program, I had no idea what to expect. I remember some of the experience of getting my MS the first time around, and some of the courses I took as an undergrad, but I wasn't sure how this experience would compare. What is an education like at a top-10 institution? Would I be able to measure up to the bar and finally put some of my constant inferiority complex to rest?

Well, we'll have to wait a while for the answer to that last one. I'm not very confident by nature, even though I know I have a modicum of programming and communication skills. I'm always looking for ways to "prove" myself, so to speak, and I tend to take on more than I can handle as a result. I am not sure that that personality trait will ever truly go away, and honestly this Georgia Tech experience stems from that whole complex - I had had a couple of bad interview experiences with companies that rhyme with Moogle and Bamazon, and was trying to figure out how to bolster my knowledge and not feel like I was incompetent. I guess this way I'll be able to point at a fancy piece of paper and say "Well, I was good enough for that".

Provided I am actually up to the task, that is.

The experience so far

Anyway, enough navel-gazing. I'm taking CS-8803 - AI for Robotics - and so far the experience has been pretty good. On the one hand the course is delivered almost entirely via lecture. I have mixed feelings about this, and feel another digression is in order. I know that there are different ways people learn, and that some people feel that they learn best visually. I also, though, have taught several classes where the student claims to only learn well visually but, if you really got down to it, what they really wanted was a step-by-step guide on how to do an assignment rather than a lecture that builds the foundations for a concept that they then apply themselves. In other words, they were (in my eyes) using the "I'm a visual learner" mantra as an excuse rather than a legitimate gripe. If you couple this with my NOT being a visual learner (I want nothing more than a big textbook to work through), I tend to be skeptical of the approach to begin with.

I have to say, though, I was pleasantly surprised. Aside from minor frustrations the lectures do a decent job of presenting the material. The way a module works is that a video of the lecture plays up to the point of a "quiz," which in a classroom setting would be the instructor asking a question of the class, and stops to allow you to input an answer into a form that is then checked for correctness. Once the answer is entered (or skipped, if you get stuck) the lecture continues until the next quiz. The lectures themselves vary widely, using video recordings in some cases, recorded whiteboard sessions in others, and still others with the professor simply talking to the students. Of most use to me were the intermittent programming quizzes - the instructor would explain a concept, then say "Ok, now you try to write a function that does this," and suddenly you're presented with a minimal Python IDE to develop code in, along with a test run and evaluation button to determine if your code is correct. It's obvious that a lot of thought went into these lectures, and it comes through in the design of the course.

Frustrations

My frustrations with the course so far are two-fold. First is that the course is programmed in Python - a language I had a passing familiarity with, but which is subject to its own quirks. If the course was in C++ or Ruby I'd have no trouble, but as it is I am learning the language as I go along. On the one hand that's good - I'll have another language under my belt when I finish the class - but it is also frustrating and increases the time I spend on assignments. I'm not sure there's any way around this, though.

The second frustration is with the video-based learning. It is extremely challenging to reference material in a video, particularly when programming. Programming is a lot about copy-and-paste, and you can't paste from a video. I have had this argument with any number of people, and I remain unconvinced - for these type of topics you can provide a video if you like, but the most useful thing to a professional programmer is a text version that allows you to 1: keep it open in a browser next to your IDE for reference, 2: allows you to review without having to hunt through a video, and 3: allows you to learn without needing to break your concentration for audio and visual stimuli. To me this is reminiscent of a problem I had at Enova, which is that for some reason the IT department thought it was a good idea to do all of their internal support tutorials as videos. Videos are an extremely poor choice when adding a VPN key, or when you want to save the information for future reference. If OMSCS is going to be mostly video-based, I suspect I'll be lamenting this for a while to come. But then again, I was able to complete the first week's assignment and understand the material (with one exception, as Bayes rule is still kicking my ass for some reason), so I guess the point is moot.

Communication

The course also has a separate message board on piazza.com, which is used to speak with your fellow classmates and ask questions of the instructor and TA. This is a great feature, I think, and has helped me a couple times. Most of the people in the course are active on the boards and very willing to help, and it is also helpful to see that some people have the same questions I do. Where I've had trouble in the course I've quickly been able to get a response on the message board, as both my fellow students and the instructors are active there.

Self-direction

One of the things I really like at the moment, but could see myself coming to dislike, is that the course is largely self-directed. If I had the time to spare, I could push through the entire course as quickly as possible. While the lectures are organized into "Lessons" that represent a week's worth of work, there is nothing preventing you from moving ahead as you work. On the plus side, this means that I was able to finish the first week's assignment well over a week before it was due, which is great as I maintain a far too busy schedule to begin with. However, I can see this causing problems near the end of the term as if we are looking at a time-based submission and grading process, I won't be receiving feedback on my submissions until they are officially "due". So for example, if I was to power through module 2 right now I wouldn't see any feedback until June 4 at the earliest, by which time the assignment will likely be out-of-mind.

I suppose time will tell on how successful this approach is for me. For now I like that I can work ahead when I have some spare time!

Evaluating my own teaching experience

Another thing the OMSCS experience is allowing me to do is evaluate my own teaching, and see what is effective. I teach college courses online for a couple universities, and seeing how this course is conducted provides some interesting contrasts. From the start, it's obvious that more effort went into the Udacity course design than goes into the courses I typically teach - which has been a constant frustration point for me as, with one exception, I often don't get to design my courses so that they have the material I feel is necessary. However, a lot of the assignments in the class I'm taking now are "did it or not" evaluations - you get points for completing the activity, but no partial points are awarded. I don't know what grading will be like yet (doesn't happen until next week), but I think "all or nothing" assignments are a disservice to the student and fail to recognize that it IS possible to half-understand something.

Conclusion

To sum up, I'm very much in a wait-and-see pattern right now. Some of my frustrations with the format may be exclusively due to this course's design - I've already heard that some of the other courses handle things a lot differently - while others may be due to the format. I guess my opinion will continue to evolve as I run through the experience. I know that's kind of a weak close to this lengthy post, but to be honest that's about how I feel right now - not very strongly on the "good" or "bad" spectrum, just "different."

I do think it will be interesting to come back and read this after I have a few more classes under my belt. So stay tuned, I guess.

Check out my latest posts at Elite Gaming Computers!

I cover SanDisk's new 4TB Solid State Drives at http://wp.me/p2yLe2-4P7
I cover ASUS' new 28" 4k gaming monitor at http://wp.me/p2yLe2-4Pb

Get 'em while they're hot!

Georgia Tech and Me

A few months ago I posted this after receiving a rejection from Georgia Tech's Online Masters in Computer Science program. Since then, after an email exchange with the dean of the school of computing, my rejection was reversed and I was admitted into the summer session, which begins very soon. I've been holding off posting about this, but wanted to give a brief touchbase. My previous post doesn't paint the program in a very good light. Some of that was due to my being hurt at an unjustified rejection, but some of my criticisms are still valid. As I get ready to register for my first course and begin taking classes online, here are a few of my observations on areas for improvement:

• Process deadlines and dates - Once admitted to the program, there are still a lot of questions. For example - I was admitted to the summer term, but I have no idea exactly what the dates for this term are. I have a vague idea of May to July, but none of the admissions material indicates the timeframe for the work I will be performing. Furthermore, the registration dates are not too highly publicized - there is a process where you can check with the registrar via the student portal, but it wasn't even really working until mid last week due to the status of OMSCS applicants. A quick email with a few reminders (alongside the DAILY emails we get with somewhat relevant general Georgia Tech news) would greatly benefit the prospective and admitted students.
• Value for money - I don't speak of the education, as I have yet to experience that (and regardless of my opinions, Georgia Tech's Computer Science program is ranked in the top ten in the nation), but rather during the application process. Form letters are inexcusable when someone has paid an application fee to be considered. This is endemic across academia - pay $50, get a form rejection. If I am paying $50, even if I don't get accepted I feel as though my purchase - which is between 5-10 hours of minimum wage earnings - warrants at least SOME individualized feedback. A personalized note. Tips for improving on the application. ANYTHING other than a generic, lazy form rejection. I understand the fears of providing a base for a discrimination suit, but in an objective measuring system designed to weed out applicants, there needs to be at least SOME room for objective feedback! I find this just as frustrating in the business world when applying for jobs (I am looking at YOU, Google and Amazon - give some freaking feedback and alleviate the single greatest complaint about your interview processes), but at least in those cases the only thing I lose is my time.
• Samples of coursework - I have absolutely zero idea how to prepare for the forthcoming coursework. Any examples of the courses themselves may have been publicized, but the accepted applicants should be reminded of these examples rather than having to resort to google searches of mass media. You're working on developing a system that will serve thousands of students each term - these convenience elements will save thousands of man-hours in help desk questions, support tickets, and overall will reduce general frustration.

Have any of you been accepted to the program? What are your thoughts on the above?

New Freelance Gig

Hey All,

I know it's been pretty quiet here lately, but I've been keeping up with my writing. I lately picked up a new freelance writing gig. I'll be writing content for the blog at Elite Gaming Computers - http://www.elitegamingcomputers.com/blog

You can see my first article here: http://elitegamingcomputers.com/kingstons-hyperx-cloud-gaming-headset/

Theme Story - Directions

Prompt this week was directions from point A to point B. I went a little weird.

First, start your water boiling. You want to get a nice, rolling boil going, so keep those flames hot. Once you've got the boil going, you'll start your journey by tossing in a handful of dirt. Doesn't matter where it comes from, but don't skimp on that handful. Add three oak leaves, with the veins removed, and two acorns. Let the mixture stew for a moment, then put on your protective clothing. Add a fourth-generation honeybee queen, being careful to keep the drones defending her from touching the boil. Using a stick from a sapling of less than two years, stir the mixture three times counter-clockwise. Then, add the saltpeter and step back – you should see a healthy boom.

If the pot survives the explosion. Instantly remove it from heat and let the mixture congeal. Scoop out three spoonfuls onto three separate cheesecloths, and place them at the corners of an equilateral triangle the size of the portal you wish to make. Make sure that the triangle points northwards – a south-pointing triangle is not something you want to experience. Step into the triangle formed by the three cheesecloths and do the hokey pokey but do NOT turn yourself around. Following this, hold your breath for three seconds and jump. If you have completed the recipe correctly, you should break through the ground into Flavortown. If not, you will land solidly on the ground and will lose your access for three years.

Theme Story - He turned the key in the lock...

He turned the key in the lock and opened the door. To his horror, he saw yet another of the damn rooms. Same pulsing green light emitting from the same pillars in each corner, same four doors pointing in each of the cardinal directions, same metal chair at the same table with – once again – nothing on it. Yet. He knew that would change soon enough, just as he knew that once he stepped forward the door would shut and lock behind him. He considered going back and trying the other doors, but after nearly fifty rooms he knew what he would find when he did so. Besides, it had only been four rooms since he had just picked a direction and went. He didn't know if he was making progress – hell, the way this place defied logic he could very well be moving in a circle repeatedly – but with no other options he decided to keep pressing onward.

He pulled out the chair and sat at the table, the screen inset into the surface once again coming to life. The same grid with the same colored dots greeted his angry gaze. He had to struggle to stop himself from tearing his hair out in frustration. He'd solved 47 of these puzzles so far. He found himself irrationally hoping that this one would be the last, or at the very least 50 would mark some kind of change in the scenario. A different room, a different puzzle, even a different light color in the glowing pillars would be preferable! Some indication that his actions were producing some effect somewhere. Anywhere.

He swiped his finger along the grid, connecting the colored dots without overlapping. This one was fairly simple, as were the others – the entire process took fifteen seconds. Upon completion, another key rose from the table. He placed the original key into his pocket with the rest of the growing collection, and palmed the new key. He rose from his chair and moved towards the door directly opposite the table - he'd taken to referring to this direction as North, but he had no real way to tell where he was even headed. He put his hand on the door handle, inserted the key, took a breath, closed his eyes, and turned the handle.

He waited a second to open his eyes, hoping to see anything different, but once again he was met with the same room. The 49^th room, with the 49^th puzzle, and the 49^th key. He sighed heavily, and moved to the chair. He plopped down with a despair as deep as his frustration had been in the last room. Fifty, he told himself. Just make it to fifty and something will change. He idly solved the puzzle on the table and grabbed the proffered key that resulted. Moving once again in the same direction, he approached the North door and inserted the key. With another steadying breath he turned the key in the lock and opened the door of the fiftieth room.

No change. Same green glow, same table, same chair, same four doors. He stepped through the door and sat in the chair, regarding the puzzle before him with a surly look. He swiped his finger around the grid, connecting the dots together, retrieving the key from the tiny slot in the table. He stood and approached the fiftieth door. He placed the key in the lock and, after a brief pause, turned it and opened the door...

… and blinked in the sudden light. Not green this time, but bright white emanating from every corner of the room. The table was round this time, the chair wooden, but the room was still surrounded by four doors in each of the cardinal directions. He crossed the threshold and a siren blared, causing him to jump out of his skin. He had no idea what it signified, but was oddly comforted by the fact that someone felt the need to signify something. A siren like that is placed for other people to hear, not for the sole occupant trapped in a maze. He approached the table and pulled out the chair. The surface lit up revealing the same grid. No, it was different this time – there was one more square in each direction, and one additional color to connect. He had apparently reached the next level. He leaned back, but oddly felt no anger or frustration. He was making progress at last. He solved the puzzle quickly and grabbed the key that emerged from the table, making his way to the North door once again. He inserted the key in the lock and turned the knob. Telling himself that if there is progress there might be an end, he pulled the door open and moved into the next white room. 

An Open Letter to Georgia Tech's Online Masters in Computer Science Program

NOTE - This situation has now changed. I leave this post here because I do not believe in shying away from my own words. I think that many of my points are still valid, the brash tone notwithstanding. Look up posts with label "Georgia Tech" for more information

To whom it may concern,

I just received my application decision, and I was turned down for admission to the online Masters in Computer Science program. I must say that I was surprised. From the FAQ on the OMSCS web site, I see the following requirements:

Preferred qualifications for admitted OMS CS students are an undergraduate degree in computer science or related field (typically mathematics, computer engineering or electrical engineering) from an accredited institution with a cumulative GPA of 3.0 or higher. Applicants who do not meet these criteria will be evaluated on a case-by-case basis; significant professional or other work experience with supporting recommendations may qualify as an adequate substitute for the appropriate academic credentials, however work experience will not take the place of an undergraduate degree. Georgia Tech will not admit applicants into the OMS CS degree program without the minimum qualifications for success. The Georgia Tech minimum criteria used in determining each applicant's eligibility for consideration shall include:

1. Evidence of award of a bachelor's degree or its equivalent (prior to matriculation) from a recognized institution, demonstrated academic excellence, and evidence of preparation in their chosen field sufficient to ensure successful graduate study; and
2. For international applicants, satisfactory scores on the Test of English as a Foreign Language (TOEFL).

In essence, the requirements applicable to me as a domestic applicant are: A bachelor's degree in computer science with a GPA of at least 3.0, demonstrated academic excellence, and preparation in the field sufficient to demonstrate successful graduate study. Let's cover each of these in turn:

A bachelor's degree in Computer Science - Achieved.
A GPA of at least 3.0 - Achieved
Demonstrated academic excellence - I have a master's degree from DePaul in computer graphics, which I completed with a 3.93 GPA while working full time. In other words, I graduated with distinction from a challenging program that I underwent while working 40 or more hours a week. I am pretty sure that demonstrates academic excellence. Not to mention that my undergraduate GPA was more than sufficient to earn me an invitation to Upsilon Pi Epsilon and would have given me the opportunity to graduate with honors, had I received the paperwork early enough.
Preparation in the field sufficient to demonstrate successful graduate study - Aside from 10 years experience as a software engineer, I have shown evidence of previous successful graduate study by successfully performing graduate studies. Setting aside my letters of recommendation, my success in my master's degree and my continuing efforts as adjunct faculty at various colleges should sufficiently demonstrate this requirement.

In short, I have met your base requirements, I have experience as a professional in computer science, and I have a demonstrated track record of academic excellence at the graduate level. So why, I ask, was my admission denied?

Let's look at your letter, which I assume was a form letter:

"Admission to the program is extremely competitive, and I am sorry to report that we are unable
to admit you"

This is the first thing that caught my eye. Here is a direct quote from the OMS CS website:

"Applicants who meet the minimum criteria will be conditionally admitted into the degree program and must pass their first two OMS CS foundational courses with a grade of B or better to be fully admitted."

One of these things is not like the other. You can't have an "Extremely competitive" admission, and then turn around and say you will conditionally admit all applicants who meet the minimum criteria. That is not how the English language works - accepting everyone who meets the minimum requirements precludes the words "Extremely competitive". Not to mention that the letter I received directly contradicted this statement (not the first time that your information conflicted during the admissions process, either). Moving on:

"While you may have some experience in computing, we do not believe that you are currently prepared for success in this program, which is extremely demanding in a broad range of Computer Science"

Some experience in "computing"? Like a graduate degree in "computing", and ten years of experience as a developer across a variety of industries? And you judged my ability in this area based on what - transcripts from a bachelor's degree? Did I not mention Big O notation, or list my favorite data structures? Perhaps it's the fact that I have been teaching students in this area for over two years - did that result from me not having sufficient computing experience, or not being prepared for success? I highly doubt your judgment on my "computing" experience when you have seen no samples of my work, and when you use terms like "computing" to describe the actual study of "computer science."

"We encourage you to take courses in Computer Science that may better prepare you for a future application of admission to this program."

How is it determined that I need to take more courses in Computer Science, anyway? Did someone test my knowledge when I wasn't paying attention? Did they see samples of my code that I wasn't aware of? What part of computer science do I need a refresher on? Should I just pick a class at random and go for it?

In short, one of two things is happening: I missed some secret requirement that I didn't meet, or my application was not sufficiently evaluated. I would like to request more information on my denial of admission, or more specific feedback (you've had three months to respond to people - a three months which you have repeatedly pushed back for questionable reasons - more than sufficient time to personalize the responses to less than 2,000 people (much less, given that this was a second round of admissions)) on how I can better improve. Were my grades too excellent in graduate school the first time around? Did I have too much employment experience?

Thanks for your time, and while I am extremely disappointed in your shortsightedness I do wish you the best of luck with the OMS CS program. I had hoped to be an advocate for this novel way of presenting the course, but after going through the application process I now have serious doubts about how it can succeed when handled as it has been.

Matt Billock

Concrete Coding Corner - Agile Estimation

In the last concrete coding corner post, I spent a lot of time talking about goals and metrics and how they relate to non-concrete endeavors such as writing and programming. One thing I deliberately left out was the obvious. Sure, I can develop any number of metrics to tell me how much progress I am making, but what do I do to tell how much progress is left?



In the business world (where most of my experiences in this area lie), companies are run on quarterly budgets that take into account resource allocation and potential projects. These projects usually have some sort of figure attached to them that represents a Return On Investment, or ROI. The goal then is to have ROI be as high as possible, while the resource allocation budget to produce that ROI needs to be as low as possible. For manufacturing processes, there are a number of factors that play into this – designers, prototyping, manufacturing, material resources, and so on. Physical costs, such as the base metal required to manufacture the body of a vehicle, are pretty much fixed. You can potentially swing a one-time decrease through intense negotiation, but you introduce yourself to potential shortfalls in quality to meet the new price. Not to mention that negotiating those kinds of reductions can be an extremely time-consuming and exhausting process.



The other aspect of any project is the, for lack of a better word, “flexible” cost. This is the cost for more ephemeral tasks, such as programming or architecture, where you cannot necessarily know how long the task will take before you begin. A project manager or product owner looks at their cost reduction options and sees this, thinking to themselves “Getting Joe to work a bit harder for a while will get this done faster, and be easier than reducing physical costs. Let's do that!” Without data to back up assertions that the goals imposed as a result are not achievable through reasonable effort, the programmer is left in the lurch – they have no power to change the situation, and thus often end up working long hours to meet the seemingly arbitrary goals.



The Problem
The primary problem with the “just work more” approach is obvious to anyone who has done creative work for any length of time – creative work is exhausting. Performing that kind of work for any length of time can feel very much like running a marathon, conflating the mental and physical worlds into a conglomeration of perceived Herculean effort. After X number of hours, with X being different for every individual, it is simply not possible to produce at the same quality level as it was possible before X hours had elapsed.



The practical upshot of this is that quality of output for creative endeavors decreases as the time spent on that task in a session increases. This is the central point that is missed when attempting to procure more effort out of employees who derive their primary work product from the expense of mental effort – longer hours means more bugs and issues, which means longer hours to fix them. It's an infinite loop of increasing required effort, which often ends up costing more to complete than if the employee had been allowed to go at their own pace.



Note that the video game industry is horrible in this regard. Long hours (60-80 hour weeks or more) are the norm in that industry as projects move towards the completed stage. Project managers in the game industry are often constrained by hard dates – such as the holiday season – that they have no way to negotiate around, and so when you cannot manipulate your end date the only thing remaining is to attempt to manipulate the amount of effort performed each week. In many cases, this results in months of extremely long hours, where programmers and artists are pushed relentlessly to meet deadlines that are forever seeming farther away. Additionally, the cumulative exhaustion of weeks of long hours seeps in, resulting in the remaining amount of work increasing despite the longer hours as bugs introduced into the system by a tired programmer required effort to fix.



Having spent time in the game industry, I wish I could say that the above was an apocryphal example, but examples abound throughout the industry, and things rarely seem to improve. The marketing and funding realities of the industry make this a problem that is tricky to solve. Some companies have large enough cash reserves that they can simply ignore the calendar and say “It'll be done when it's done,” but most others need the infusion of cash provided by a completed project to continue to do things like pay rent, electricity, and employees (often in that order).



Analyzing the issue
The game industry has a situation that arises from external constraints, but it also is a byproduct of creative work – there is no way to know how long a task is going to take before it is started. Sure, you can have a ballpark. For example, I've written two books so far, and each took about 30 days for the final draft. Given that, it's reasonable to assume that if I were to write a third book, it would also take about 30 days for the final draft.



Except when it doesn't.



This is the most frustrating thing about programming to non-programmers. Two tasks can seem extremely similar, but one may take five minutes while another would take a week. The reasons why are numerous – fractured code bases, incompatible patterns, and so on – but ultimately there is no obvious reason that a layman can point to and say “Oh, that's why that took so long!” This problem, when repeated multiple times over the course of a project, can sour the relationship between a project manager and a programmer, taking what was originally a productive and respectful interaction process and making it confrontational. It is usually not the result of any intentional slight by the programmer or intentional misinterpretation by the PM, but instead it's an issue of trust.



Every time a programmer claims that a task will take longer than originally expected, or that they have been delayed, the PM has no choice but to assume that the delays are intentional as they have no insight whatsoever into the development process. This may not be an overt assumption, but it certainly occurs on some subconscious level and factors into decision making and interactions. This injects stress and discord into the process, which builds up over the course of a project and results in negative perceptions. A project could be on the most efficient possible track to completion, making strides that no other team could possibly make with the same material, but once the perception is soured that effort will never be seen as sufficient.



Addressing the Problem
The problem, as we've already mentioned, is that the PM has no way to know when a given task will be completed. The frustrating thing is that oftentimes, neither does the programmer. Sure a programmer can make an educated guess, but there are always factors outside of their control that play into the end result. Thus, the first approach to addressing the problem is to rely solely upon the guesses of the programmer. The programmer takes a look at a task before he begins, and gives an estimate of the effort required to complete that task. As a project manager is concerned with dates and timelines, they often push for time-based estimates (“It will take four hours”), and many programmers agree to the system.



Problem Solved, Right?
Once a unit associated with time has been decided upon, everything very quickly goes to hell. If a task that should “only take four hours” suddenly takes 40, then there is a perceived deficit. A project manager looking at a project plan sees the extra 36 hours of effort and is forced to conclude that “The project is now 36 hours behind.” This amount grows or shrinks as effort continues, and the perception of the project manager changes each time. If a task takes less time than expected, then the programmer is “making good progress” and the project is “on track.” If a task takes more, then the programmer is “introducing delays” and the project is “delayed.”



Notice the attribution of effort here. This all ties back to the choice of time as a base unit. Once you are estimating in terms of hours, the natural human tendency is to start watching the clock. We've been trained to pay close attention to time periods – 8 hours of work, 1 hour for lunch, 30 minutes to commute, and so on – that we subconsciously monitor the amount of time taken. When something takes longer than stated, it is frustrating and unsettling. By tying the estimates of a programming task into this time-based system on a granular level, we introduce the same predilections and perceptions into the evaluation of progress. It removes the ability to look at a task objectively and say “we are making X amount of progress.” Instead, everything is cast in terms of time. Time elapsed, time remaining, time behind, and when looking at the time associated with a project subjective judgments are added into the project status. Choosing an estimate in terms of time has resulted in looking at the project at a micro level, removing the ability to see the forest through the trees. The subjective perceptions of “late” and “early” that have been pounded in to us from our youngest years come into play, and color our perception of the project's progress.



Aggregate Data
One of the reasons behind this is that it is far too easy to look at a project as a series of finite tasks. Once these tasks have a discrete time associated with them, it is far too easy to take each task as an element in isolation. Good project managers, though, recognize that each task is only part of a larger whole. They strive to look at a project in aggregate – that is, as a combination of all of the tasks. They see the four hour task taking 40 hours, but then realize that the project is still 25% completed with one quarter of the budgeted time elapsed.



Thus, we see the path to our solution. An individual four hour task may run long, and may result in a delay. But on average, four hour tasks will typically take four hours. Some may be shorter, some much longer, as that is the nature of creative endeavors. However, looked at over a span of time, it is possible to develop a translation. If the average of all “four hour” tasks is 4.5 hours, that gives us a reliable predictive measure. We now know that if the programmer estimates a task at 4 hours, it will take 4.5 hours on average. This allows us to adjust our estimates and determine where the project actually lies.



A Problem of Units
At its core, with this translation table we've improved upon the primary concern – developing accurate estimates of project effort. However, we haven't addressed the soft factors of the issue. By associating progress with hours of effort, we are still tapping into that primal clock-watching urge, and thus still adding in the subjective judgements that go along with it. This is the problem with any time-based estimate system – it is tied to a known and familiar system, and even though it may operate on a varying scale, there is a subconscious perception of variation that permeates the project, regardless of the intentions. Because “hours” was chosen as the unit, this is tied back to real world elapsed time, and we end up with a slightly improved version of the same problem.

As it is, we now actually have the seeds of a potential solution. We've identified that looking at effort in aggregate provides us with more accurate estimates of project completion. We've also identified that expressing estimates in terms of time introduces needless subjective judgments into the process. We finally know that despite our best efforts, we do eventually have deadlines to meet – we cannot continue to just work forever.



A Different Approach
What we are really doing when estimating a task is building a view of the entire project. A project is the sum of its parts, and having broken a project down into constituent tasks it thus follows that a project's total effort is the sum of the efforts of all of its tasks (it is more complicated than that, but usually the point holds well enough). So by adding up all of the estimates associated with a project, we can determine a total estimate for the entire project. In a time-based system, this is expressed in terms of hours and days.



However, at this point we are working simply with numbers. We've established that due to the nature of creative work estimates are by definition non-concrete. So instead, we apply analytics to find out what “four hours” actually means in terms of calendar time. At this point, we're simply doing number analysis – averaging numbers together to produce a measure in terms of time. Since we have the same units on both sides of the equation, it follows that when these values on either side of the equation differ, then one of the choices of units is thus incorrect. It should be obvious that 4.5 hours does not equal 4 hours, so one of those figures must not really be “hours” as we know them.



Let's instead refer to the right side of that equation as points. Instead of 4.5 hours = 4 hours, we have 4.5 hours = 4 points. By removing the hours from the right hand side of that equation, we've suddenly addressed the subjective issue with our measurement system so far – we took away the time base in order to find the actual value, and as a result hit upon a number system that allows us to assign arbitrary values to tasks that develop meaning as a project progresses.



Building upon this gives us a more reliable means of estimation. Instead of assigning an hour estimate to each task, we assign a point value. The point value is indicative of the general effort required to complete the task, just as the estimate in hours was, but by removing the time value we remove subjective expectations. We assign higher numbers to challenging tasks, and lower numbers to easier tasks, knowing that on average the effort expended for each classification of task will average out to the same amount of effort, even if one specific task takes longer than another.



Tying it All Together
As we noted earlier, deadlines are inescapable. We cannot simply work forever. Thus, status reporting becomes a matter of determining how much longer the project will take to complete. We have a method of assigning difficulty to our tasks by giving them point values, and the point values are arbitrary units and thus remove all subjective judgments and perceptions. So our task at this point is to tie these points back into time periods.



This is actually what we were hoping to accomplish with our original estimation scheme – determine the amount of time it will take to complete a project. We simply hit upon the wrong units, and thus added a lot of extra cruft to our project plan. By trying to tie points back to hours, we become too granular – the points completed per hour varies far too much from task to task to give a worthwhile measurement. So the only option is to expand our averaging time window. This becomes an arbitrary choice, but I've seen the most success with week-long and two-week-long periods, known in the agile methodology as “Sprints”. You add up the points completed during each sprint, then keep a running average of the efforts completed. In doing so you tie the point values back into time values while also accounting for the aggregate variance of the data. For example – if I have a project that is has 170 units remaining, and I have been averaging 17 points completed per one-week sprint, I can say with a fair degree of confidence that there are 10 weeks remaining in the project.



This 17 number – the average point value completed during a sprint – is known as “velocity.” A project's velocity is used to relate the overall project duration back to the time required to complete it. By increasing the unit of measurement and divesting our estimates from subjective measures, we've introduced a more reliable system of estimation – simply divide points remaining by velocity. What this does is twofold. First, it gives you a quick and dirty estimate of the time remaining in a project that, over several weeks, tends to become very accurate as the amount of data to measure increases.



The second thing that it gives you is the ability to say no. Take, for example, a project that is starting to trend towards long hours. If you archive your data properly, you can point to previous examples where such an approach only made matters worse. Instead of “well, that will be pretty hard on the team,” you can instead say “We tried this before, and added 50 points of work while only achieving 47 from the increase in face time. How do you think it will be different this time around?” By looking at the point value of each task to be completed, some negotiation can take place and instead of giving subjective measurements and impressions, you can provide specific feedback with actionable data. Of course in some cases this won't do a damn bit of good, as the project deadlines may be particularly unforgiving, but in aggregate this approach should produce an environment that is collaborative and built upon a reliable history, as opposed to a relationship founded on distrust and fostered in a confrontational environment.



What does this mean for writing?

Ultimately, this method obtained from programming can be applied to any creative endeavor that can be assigned a subjective judgment of “easy” or “hard.” Simply break a project down into constituent tasks, then assign a point value to each task representing its difficulty. Of course after a certain point the tasks become repetitive – how many ways can “Write this scene!” be phrased, after all? However, one thing I have found when applying this approach to my own efforts is in support of my previous post on metrics – once you start putting your mind to measuring your progress in a more objective manner, you are able to get a better feel for how things are actually going with the project. While you may or may not have deadlines to work towards, and you may or may not have people waiting on your progress, the concrete proof of your progress is often enough to help you break through any bumps on the road, and stave off periods of despair over how long a given project is taking.

Theme Story - A Milestone Birthday

Brian took another swig of his drink, grimacing as the cheap bourbon set his mouth on fire yet again. He preferred the smoother Irish whiskeys, but in a bar in Kentucky your drink choice was more or less a foregone conclusion – particularly when someone else was buying. He looked over his glass at John and, catching his eye, raised it in the traditional salute of thanks. Niceties achieved, he set the beverage down to give the ice some time to melt and mellow the flavor.

Thirty. He couldn't believe it had been so long. By some accounts his life was half over, and even the most optimistic estimates only reduced the deduction to thirty-three percent. Sixty years if he was lucky, of which the last fifteen to twenty were likely just biding time. So call it forty years reliably. He took another swig – too soon, as the heat on his tongue was happy to inform him – and tried not to dwell on how he was spending yet another night punishing his liver in the same tavern. Perhaps if they had tried to class up the place a bit he could have stood the monotony – sand some of the walls smooth, maybe stain the wood a darker color than the tannish-brown that didn't so much reflect the light as absorb it before casting it out into the world. Of course that would offend the clientele, the hearty salt-of-the-earth folk who liked their bars slapdash, their camaraderie boisterous, and their whiskeys blazing.

And their music twangy, as Brian was reminded by the jukebox happily blaring the latest blues-inspired nothing from a country-western group whose last chart-topper was released a decade ago. His melancholy allowed him to scoff at the clear efforts of musicians past their prime, but his past years in the country had helped him to develop an appreciation for the style, finding some small comfort of home in the chord structures if not the banal lyrics. He'd tried once, a couple years ago, to get the jukebox to do more than lament the lives of cheating boyfriends and lost tractors, but the death glares that he'd earned as the opening bars of the one Rush tune that was available on the machine played had been enough to abash him of any future attempts at culture.

He took another swig, this time more satisfied with the spirit. Sure the ice watered it down a bit, but sometimes that was the only thing that made the drink tolerable. He smiled to himself as he remembered the hazy days of college, the quantity-versus-quality binges Friday after class. Had that really been nine years ago? It was odd – the gray hairs didn't bother him at all, but when Brian realized how long it had been since he'd been a student he shuddered with the weight of age. He still felt like that wayward scholar most days, no idea where he was headed but doing his best to not appear that way. Somehow fake-it-til-you-make-it never made it past step one, and one of his more pervasive ridiculous fears was that someone would expose his attempts at maturity for the pathetic facade that they were.

Someone asked him a question, and Brian took a second longer to process than was proper. With a small shake of his head he gave a non-committal response, and received a hearty clap on his back to go with the chuckles at the birthday boy who was showing his inebriation so soon after the party's start. Conversation resumed, and Brian went back to staring at his glass. Was this what forty would feel like? Fifty? He went to take another swig, but suddenly noticed that the glass was empty. Setting it down, he leaned forward on his hands as he pondered.

Thirty years gone. Seven years a professional, and what did he have to show for it? A slice of the American dream, sure. For what that mattered, at least. The small house on the edge of town was often a harsher task master than his manager, with the grass that never stopped growing and the vermin that never stopped burrowing. A job that was decidedly middle-class, which in essence meant he was able to come out enough ahead every paycheck to set aside a meager amount for those waiting years in the coming autumn of his life. No loving wife, but a parade of women of increasing age, with accelerating desires and relationship goals that never seemed to align with his long-term plans. Plans towards which he had, so far, made zero progress.

Another contained puddle of bourbon had materialized at his elbow, and he gave a grateful smile to Katy – the kind purveyor of the current round, and even kinder woman who was the most recent of those trying to find a place to fit in with him. Not that he had women lining up, but rather he understood what they were looking for and how to provide it. He took a sip and shook his head. That last bit was unnecessarily harsh. Katy was sweet, and even if she felt the same pressures that had been put on him by others of the fairer sex she gave no indication of the pushiness that others had been all to happy to unlimber on him once things started to show signs of progress. He thought he appreciated that more than anything – sure she was looking for a future, but she wasn't using the strong-arm to make it happen.

Brian took another swig, smiling again as raucous laughter filled the table in response to some joke or another. He'd never been much for birthdays, he realized that, and his friends seemed to tolerate that well enough, but he knew he was on the edge of stretching patience. He grit his teeth, smiled, and jumped into the conversation – feeling out of place and awkward, but still knowing what was expected of him and fulfilling those expectations as best he could.

Perhaps the next thirty years would hold the wisdom he seemed to be missing, or the sudden achievement of greatness. He sipped the bourbon, made a random joke. Doubtful, but he figured he'd give 'em his best shot.

Theme Story - Superstitions

Mack crept slowly through the doorway, squeezing between the unfinished stone surface of the door and the doorframe and entering the great hall. And that it was a great hall was unmistakable – soaring columns at regular intervals along either wall led off into the distance, carved with exquisite detail in direct contrast to the unfinished rock wall that gave entry to this room. Though he could not see the ceiling in the murky, flickering light thrown off by the line of braziers down the sides of the room, the echoes of his footsteps as he crossed the polished marble floor told him that it certainly lay far above him.

He looked about warily as he moved, expecting an attack at any moment, but saw nothing save a free-standing mirror in the center of the great hall. It glowed with a green light, casting a sickly pall that seemed to negate the effects of the flames – cold where the firelight was warm, static where the firelight flickered, chilling where the firelight was comforting. Mack saw himself in the mirror as he approached, his steps showing more confidence than he felt. The light gave his jovial features a grim cast, his bright red beard looking almost black underneath an unearthly pale face.

“So you have finally come.”

The voice seemed to come from everywhere, assaulting Mack at his very core as it battered his ears. He looked quickly about, but could see no source for the sound in the room. A flicker of movement caught his eye, and he turned back towards the mirror. His reflection seemed darker somehow, as though something blocked the light approaching him. Mack squinted, and was able to discern a ghostly shape in the shadow. As he watched, the shadow in the mirror solidified. Darkness gave way to light as features emerged – a head took shape, a hawkish nose growing out of its center, and a dark body became ornate robes with glowing embroidery.

Before long Mack was face-to-face with a figure that was not there, seeing only a reflection in the free-standing mirror that should not be. A tall, pale man looked back at him. His robes rippled in black and green, with embroidery along the collars that glowed with the same unearthly green light as emanated from the mirror. Dark eyes looked out over a long nose, but no hair framed the eyelids, brow, or head. Most disturbing was the complete absence of a mouth – between the hooked nose and the strong chin lay flesh without blemish – a solid sheet of white that jarred Mack's sense of reality.

“I have watched your progress through my domain.” The voice continued to beat at Mack, gravelly and rasping. “You are the first person to reach my chamber in twenty years.”

Mack peered around him, looking for both the origin of the voice with no source and the body casting the impossible reflection, but saw only the sickly light of the mirror fighting back the firelight from the surrounding hall. He turned back to the mirror. “Who are you?”

The voice chuckled. “Seems as though I should be asking you that question. You are the invader here, after all.” The reflection heaved a sigh, then the voice continued. “When this land was young, and your people were emerging from their caves across the oceans, I was worshiped as a god. When you had taken your first stumbling steps, I had been here for a thousand years. Ten thousand.”

Mack shook himself free of the aural assault. “You're the Destroyer.”

The figure nodded, though it had not been a question. “Where there is good in this land, I bring evil. Where there is light, I make darkness. Where there is peace, I bring war.”

Mack pulled a stone from the pouch on his belt, sliding it into his sling. The weight felt odd, heavier than a stone should be at that size, but he only noticed this in passing. “Not anymore. Show yourself, Destroyer.”

The mouthless figure looked at him intently. In his mind's eye, Mack saw an evil grin but the smooth patch of skin on the creature's face remained unchained. “But I already have.”

Mack whirled about, looking for the source of the reflection, but an empty hall met his gaze. He turned back to the mirror. “You lie. I command again – show yourself, coward!”

The figure raised a white hand, nailless finger extended towards him. “You command nothing here.”

The Destroyer made a jabbing motion, and Mack flew backwards. He landed in a heap on the marbled floor, sliding to a halt. Pain emanated from his chest as though he had been hit with a battering ram. He scrambled to his feet and looked around, readying himself for the next blow, but still saw nothing to defend against.

The Destroyer laughed, the sound beating against Mack's sanity. “I am sad to say that your journey is at an end, today.”

Its hand swept to the left, and Mack went sliding with it, crashing into a column. He stood, only to be knocked from his feet again at another gesture of the creature. The blows began to come more frequently, tossing him about as though he were a rag doll. Mack took bruises from the columns and floor, bouncing off of them as he was sent careening around the space. He did his best to absorb the impacts, but could not keep up with the sheer volume.

Seconds stretched into minutes as he was dashed into the columns and floor, until suddenly the motion ceased. Mack spat blood as he rose to a knee, breathing heavily and wincing at the pain of cracked ribs. He rose and slowly approached the mirror again, watching the Destroyer's reflection warily.

The figure in the mirror laughed again. “Oh, what fun you have brought me.”

Mack wiped a hand across his face, clearing his vision and some of the pain. “That is not all that I bring you, beast.”

The Destroyer cocked its head. “You have a tribute?”

Mack nodded. “And per the old code, you must allow me to make my offering.”

The figure waved a dismissive hand. “Do not speak to me of my obligations, mortal. Present me with your gift, and then we shall return to your lesson.”

Mack smiled slowly and began to swing his sling in a circle. He opened his mouth and spoke the words he had been given. “I have journeyed far, with this piece of a star. Fought battles grand within your sickly land. Now I strike the killing blow at the beast that does not show, creating fortunes poor as I close the door.” As he completed the verse he released the end of his sling and the special stone flew outward. Time slowed as it crossed the distance to the mirror, the small crystals on the star stone glowing with a red light.

The Destroyer's eyes opened wide as the stone reached the surface of the mirror and, with a loud noise, bounced off the surface. It hit the ground just in front of the mirror, rolling to a stop at Mack's feet. The laughter came again, and the voice assaulted him once more. “It seems as though your gift has been rejected.”

Mack smiled. “Has it?”

The Destroyer began to raise a hand again, but a thunderous noise interrupted his motion. A great crack appeared in the surface of the mirror, cutting across the middle of the creature's body. The Destroyer shook itself as though it had been struck, but before it could recover another peal of thunder came, and another crack, quartering the visage. Mack heard a scream begin in his head, an anguished howl scrabbling at his mind, but the sound was silenced as the mirror shattered into a thousand pieces. The shards sparkled in the firelight as they fell to the ground, the unearthly green light extinguished. As the last piece struck the ground, a tremendous wind arose in the hall. It howled through the columns and echoed under the vaulted ceiling, sweeping the darkness before it as it traversed the room.

A single ray of light stabbed down from the ceiling at the far end of the hall, then another, and still another as they marched toward Mack from the distance. The light reached him and he shielded his eyes against the sudden glare, feeling the room heave beneath him. His eyes adjusted after a moment as the rumbling died down and the wind faded, and he opened them to gaze upon a transformed hall. Gone were the murky firelight and shadows. In their place polished and ornate stone glittered in sunlight, coming in through great square openings in the vaulted ceiling.

Mack basked in the warmth for a moment, reveling in the satisfaction of his victory. He took a step forward, and promptly placed his foot directly on the star stone. The stone skidded out from beneath him, taking his balance with it as he crashed awkwardly to the ground. Mack shook himself and rubbed at his hip, which had taken the brunt of the fall. He looked back at the shattered mirror, and winced. Seven years was a long time, but he was ready to face the bad luck. He put a hand down to help himself up, and winced again as a piece of the mirror penetrated his skin. He just hoped he would survive the ordeal.

Theme Story - Fingers Crossed

“Keep your fingers crossed.”

Those were the last words she had spoken, over a month ago. Steve had considered going to the police – several times, in fact - but with the way things were playing out in the streets he suspected that the cops had better things to do that track down a wayward wife. Steve figured that she had found a spot on a boat, and taken her chances on her own.  She certainly hadn’t returned with any of the supplies they had so desperately needed. Steve idly wondered whether the furtive whispers he’d overheard were true. Stacy was still a comely woman, though she was pushing forty. He tried to think the best of her, but the worst of times seem to bring out the worst in humanity.

Steve walked into the kitchen and grunted at the meager remains. True, Stacy’s departure had left him and Evan just barely enough to get by on, but they were down to their last couple pounds of rice, only a bag or two of dehydrated beans remained sitting on the counter, sagging as though in defeat. He walked past the counter to the window bringing the only light into the galley kitchen, watching the pandemonium as best he could.

The water had crested the docks yesterday. Steve did a quick bit of mental arithmetic and realized that if the water continued to rise this quickly that his tiny apartment would be beneath the surface in just over a week. He could already see makeshift boats in the streets, people doing anything they can in their desperation to get to safety. They wouldn’t make it far, of course – the last of the real boats had left over a month ago, back before the planes had stopped flying – but that kind of logic was lost on a person desperate enough to take two doors lashed together out to sea.

He heard a small shuffle behind him in the two-bedroom apartment, but didn’t move from his perch. The footsteps trailed across his hearing as they moved over the hardwood, registering but failing to penetrate Steve’s thoughtful despair. It wasn’t until the tiny tug he felt on his pants that he looked down to see Evan, bright and chipper with the energy and innocence only a five year old can show.

“Is mommy coming today?”

Steve shook his head. “Not today, buddy. She’s still out.”

“She must be bringing back a lot of food,” Evan concluded with a child’s confidence. “I hope she hurries. I want that pizza.”

“Me too, buddy,” Steve agreed absently. He watched as two boats in the street below collided, sending their respective loads to the asphalt below as the owners took out their shared frustration on each other. He turned away before the inevitable escalation, not wanting to see yet another killing in the street. Steve dropped down to his knees, putting on a smile and looking Evan in the eye. “What say we play a game of checkers before breakfast?”

Evan’s eyes lit up as he giggled with anticipatory glee. “Yes! I’ll be black this time!”

Steve smiled and nodded. “Sounds good to me. Go get the board, buddy.”

Evan raced off as Steve straightened up, taking one last look out the window. “Good luck, Stacy,” he whispered, “wherever you are.”

Theme Story - Message in a Bottle

(Prompt this week: you are stranded on a deserted island with only a bottle, a pen, and a piece of paper)

John Jacobsen

200 Palm Tree Lane

Some Deserted Island, The Ocean

To whom it may concern,

I have spent the better part of a year trying to figure out exactly what to put here. At first I thought that putting everything I could remember about my location would be beneficial, but seeing as I was asleep during the flight that went down I realized I had nothing to add. I then thought that I would put a note to whomever might find this, asking them to care for my loved ones and contact my family, but that’s a gamble at best. I suppose you can consider this a cry for help, and if I knew what date it was I’d put this on the letter so that some scientist somewhere could use some estimate of the ocean currents to deduce the range that my little home lies from the nearest land mass, but ultimately I’m writing this under the guise of a single realization.

I’m never getting off this island.

They say hope springs eternal, and that the last thing you should lose is faith. I can’t say they’re wrong, and maybe I have hit rock bottom. All I know is that I am stuck here, and you are… wherever you are, and that we will probably never meet. And to be honest, I don’t see a lot of difference. Here I am driven mad by routine, forced into action by the basic right to live, dealing with conditions that I would prefer never to have seen in the first place. There, you likely are the same – driven to action by a need for shelter and food, finding solace only in the few spare moments the overlords of circumstance see fit to allow you to pursue the things that truly hold meaning for you. Here I am surrounded by wildlife unable to comprehend my motives or, sometimes, my very existence. There, you are surrounded by the self-centered, possibly aware of your motives but more likely resistant to them, your existence seen as a nuisance at best.

I don’t know where I am, but over the past year I’ve slowly come to the realization that I honestly don’t care. Same shit, different day. At least this time I get to spend my leisure moments relaxing on a beach.

Send help if you can. If not, no big loss.

John Jacobsen.

Writing Prompt/Theme Story - First Sentence and Time Travel

(This time I combined the theme post with my writing prompt. Not sure if I like the result or not - I'll let you be the judge)

The air was calm. The water was still. The insects in the surrounding jungle had stopped making a sound. It was like everything had just stopped all at once.  Jason looked around, mouth agape at the sudden silence – the deafening cacophony of nature, which had been his constant companion for the past two months, had never once abated. He took another step into the clearing, and tentative sounds emerged once more. Each step towards the obelisk in the center raised the decibel level, until by the twentieth step the sound was back in full force. Vague discomfort faded, and Jason stood examining the object in the center of the clearing.

It stood in the exact center of a perfect circle cut in the vegetation. Vines, trees, and undergrowth abruptly gave way to waving Kentucky bluegrass, itself an oddity in the rainforest. The obelisk stood in the center of the tranquil sea, its alabaster body covered in complex carvings and symbols. The flow of the symbols tugged at Jason, hinting at a meaning beyond his grasp – a word on the tip of his tongue just begging to be heard. Jason moved closer to the obelisk, examining the carvings. Smooth lines festooned the surface, showing no evidence of tool work common among ancient relics in the area. The sunlight reflecting off the surface was unbroken, marred by neither crease nor gouge. The craftsmanship rivaled the greatest of modern fabrication techniques, the perfection bespeaking exquisite care and skill on the part of the craftsman.

Jason reached a tentative hand out to the object. His fingers lightly brushed the surface before he jerked them back, shaking feeling back into them. His arm hairs stood on end, static electricity coursing through him as he reached forward again. This time, there was no shock – his fingers connected solidly with the object. What appeared smooth at a glance felt pebbled as he ran his finger down the side, exploring the texture. His wandering finger found one of the designs – an odd spiral that folded back on itself three times. He began to idly trace the design, his finger moving of its own volition along the curves of the engraving.

The sounds surrounding him began to die out again, but this time a low pervasive hum began to rise in their place. Jason’s eyes widened at the change, but his finger continued to walk the curved path, increasing in speed with the volume of the basso rumble. It came from everywhere at once, shaking Jason’s bones, causing his teeth to chatter uncontrollably, but his finger continued to push onward. His arms visibly shook with effort as he tried to pull away, but he was unable to stop the progression.

At the halfway point in the design the hum became a high-pitched whine. The ear-piercing shriek appeared out of nowhere, starting as abruptly as the basso rumble had stopped. The greenery of the clearing faded as his finger moved, occluded by a white glow originating from the obelisk itself. The intensity grew as he moved down the spiral, becoming blinding while still allowing perfect sight. The world faded into a white light, the obelisk only making itself known through the pebbled texture under Jason’s finger.

Jason’s finger finished its traversal, and everything stopped at once. All the jungle sounds, the whine, the light – all ceased to exist. He blinked his eyes, seeing the obelisk as an after-image against his eyelids in the green and purple of photo-negatives. He opened his eyes to look around, but was met with only darkness. There was no breeze, no odors, no light, no sound – the entire world had ceased to exist. Time dilated as Jason stood in the nothingness. His finger hadn’t moved, but he could no longer feel the obelisk. Minutes passed like seconds, and seconds passed like hours. With no feedback, no frame of reference, an eternity whipped by in an eye blink.

A sudden buzzing caught Jason’s ear. There was a bright flash, and then a rising glow. Bars of light, floating in empty space as they increased in illumination, pierced the blackness. After a few moments details appeared, the soft white light reflecting off matte gray as his surroundings came into focus. Jason blinked a few times, clearing the afterimages as his eyes adjusted to the new light. The obelisk was gone, as was the grass. Unmarked gray lined the walls, floor, and ceiling of the room he found himself in, broken only by the bars of light along the floor of the chamber.

Realizing he was still holding his finger out he snatched it back to his side. He checked the finger, but saw no damage imparted by the strange object. His breathing quickened as he looked around, all signs of the world he knew gone. Jason moved about the room, running his fingers along the unbroken walls, but the only variation in the perfect octagonal enclosure was the light emerging from the floor.

“IDENTIFICATION.”

Jason jumped at the sudden sound. The voice came from everywhere and nowhere, deafening, permeating his very being. He clapped hands over his ears in shock, but the sound had left before he completed the maneuver. The silence rang in his ears, stretching as no further information came. Jason swallowed, and spoke to the room.

“Hello?”

His voice sounded tentative, the sound small in the acoustically dead space. He waited for a response, but none came. He had just opened his mouth to try again when the voice invaded his body once more.

“IDENTIFICATION INVALID. RESTATE.”

“I, uh, I don’t have any…” Jason’s voice trailed off as he stumbled over the words. “Who… who are you?”

Once again the silence stretched out, and Jason stood silently. After another pause, the booming voice returned.

“IDENTIFICATION INVALID. UNAUTHORIZED ACCESS DETECTED. ASSISTANCE REQUESTED. PLEASE STAND BY.”

The words hit him with a solid force, the meaning both plain and confusing. Jason looked around at the walls, seeing neither speaker nor microphone.

“Uh hey, what’s going on here?” His voice had a bit more strength behind it, but the complete lack of ambient noise made it continue to sound thin. “Where am I? Who are you?”

He waited expectantly, but no reply came. After several minutes, he began to pace the room. He ran his hand along the gray walls, but found nothing. Each edge of the octagon appeared to be six paces long. He bent down to examine the light source, but could detect neither power connection nor method of generation. He stood up and struck at the wall, balled fist bouncing harmlessly off the hard material. He shook his hand and cursed, checking the bones tenderly but finding no damage. After another moment, he returned to the center of the room and sat down.

“Who are you?”

The voice startled Jason, and he turned around to look at the source. A dark figure stood outlined in the center of one of the walls, bathed from behind in bright light. The voice was feminine, but the shape could have been anything – amorphous, columnar, indiscernible. Jason squinted, trying to discern details.

The figure shifted slightly. “Can you understand me?”

Jason nodded. “Yes, I can.”

“Good.” The figure stepped into the room, revealing a woman’s head over flowing purple and maroon robes. The robes obscured the woman’s figure, the lack of lines and smooth complexion of her face bespeaking youthful vigor. “Who are you?”

Jason stared at the woman. “I’m, uh, Jason MacIntyre.”

The woman frowned. “I don’t know of anyone by that name authorized to use this facility.”

“Facility?”

The woman ignored his question. “How did you get here, Jason MacIntyre?”

Jason shrugged. “I don’t know. I found this carved pillar in a forest clearing, and now here I am.”

“Hmm.” The woman cocked her head, as though listening to something, then abruptly focused on Jason again. “Please come with me, Jason MacIntyre.”

“What? Where are we going?” The woman either ignored him or didn’t hear him as she turned and stepped back into the light. After a moment’s indecision, Jason got to his feet and followed her. She walked confidently down the brightly-lit corridor, white and featureless walls glowing with a daytime intensity. Jason caught up to the woman. “Excuse me, miss. Where are we?”

“All will be made clear soon enough, Jason MacIntyre.” The woman’s voice lacked inflection and emotion, sounding as mechanical as an electronic answering service. Jason shrugged and continued to follow the woman.

The hallway abruptly ended, leaving the pair standing in a corner. As Jason opened his mouth to ask another question, the woman raised a hand against the wall before them. A circle flashed green around the woman’s hand and the wall simply faded, revealing a small room beyond. Along the far wall was a man sitting at a desk, staring into space. The desk was made of some featureless gray material, similar to the walls in the earlier octagonal room. The woman gestured, and Jason stepped across the threshold. He turned to look at the woman for instruction, but the wall had already reappeared behind him. Jason stepped forward, putting a hand to the wall in wonder, feeling nothing but solid material beneath his hands.

“It won’t work for you, at least not yet.” Jason turned to look at the man behind the desk, who had lost his unfocused look as he eyed Jason appraisingly. He indicated a small chair opposite the desk. “Please have a seat, Mr. MacIntyre.”

Jason moved over to the chair, examining it suspiciously. He reached out a hesitant hand, but felt nothing out of the ordinary. After a moment’s hesitation, he sat down and looked at the man across the desk. “Who are you?”

“You can call me Frank.” The man smiled. “And you are Jason MacIntrye.” Jason nodded. “I bet you’re wondering what’s going on.”

“You could say that again.”

“Well, unfortunately I wish I could tell you that. Your arrival has caused a lot of concern.”

“Arrival?” Jason looked confused.

Frank nodded. “The octagonal room – that’s our arrival area. It’s been closed off for nearly twenty years. I’m guessing some technician forgot to disconnect the power.”

Jason stared blankly. “Ok, wait. What’s going on here?”

“This is going to be a bit hard to take.” Frank sighed. “Mr. MacIntyre, you’ve travelled approximately six hundred years into your future.”

“Wait. What?”

“The device you found was one of our early experiments in time travel. By activating it, you activated a recall beam that pulled you from your time into ours.”

“Time travel? Recall beam? All of that’s impossible!”

Frank chuckled. “I’m sure it seems that way to you, but I assure you it is both possible and feasible.”

“But what about Einstein, and all that relativity stuff?”

“Ideas of science evolve, Mr. MacIntyre. A few thousand years ago, people were convinced that Aristotle’s four elements were all that composed the cosmos. Not all that long before your time, people were convinced that it was bad blood that caused disease. Let’s just say our understanding of physics has changed in the intervening time.”

Jason exhaled deeply. “Huh. So, six hundred years?”

Frank nodded. “Give or take.”

“Where am I, then?”

“You are in a small research facility near what you would have known as Des Moines, Iowa.”

“Would have known? Was there some sort of massive war or something?”

Frank chuckled. “Several, actually, but really we’ve simply evolved past our need for physical delimiters of space.”

“Physical delimiters?”

“Locale designations, addresses, and so on. Those hold very little meaning these days.”

“So how do you know where you are?”

Frank shrugged. “Call it something like GPS. With the advent of quantum teleportation, we simply refer to everything by coordinates. It’s easier and produces a more accurate description of location.”

Jason frowned. “Teleportation? Like Star Trek?”

“Star Trek?”

“Never mind.” Jason looked around at the room, then back at Frank. “Seems kind of sparse. Does no one decorate in the future?”

Frank chuckled. “Oh we do. We just have different ways of seeing it.”

“Different ways?”

“For lack of a better way to put it, every person has a computer built into their brain that constantly affects what they see.”

“Computer built….”

Frank waved a hand. “Look, we could spend years talking about the things you’ve missed. But in the end, right now we have two questions to focus on.”

Jason cocked his head. “And those are?”

“How did you get here, and how in the hell will we ever get you back.”