Concrete Coding Corner - Agile Estimation

In the last concrete coding corner post, I spent a lot of time talking about goals and metrics and how they relate to non-concrete endeavors such as writing and programming. One thing I deliberately left out was the obvious. Sure, I can develop any number of metrics to tell me how much progress I am making, but what do I do to tell how much progress is left?



In the business world (where most of my experiences in this area lie), companies are run on quarterly budgets that take into account resource allocation and potential projects. These projects usually have some sort of figure attached to them that represents a Return On Investment, or ROI. The goal then is to have ROI be as high as possible, while the resource allocation budget to produce that ROI needs to be as low as possible. For manufacturing processes, there are a number of factors that play into this – designers, prototyping, manufacturing, material resources, and so on. Physical costs, such as the base metal required to manufacture the body of a vehicle, are pretty much fixed. You can potentially swing a one-time decrease through intense negotiation, but you introduce yourself to potential shortfalls in quality to meet the new price. Not to mention that negotiating those kinds of reductions can be an extremely time-consuming and exhausting process.



The other aspect of any project is the, for lack of a better word, “flexible” cost. This is the cost for more ephemeral tasks, such as programming or architecture, where you cannot necessarily know how long the task will take before you begin. A project manager or product owner looks at their cost reduction options and sees this, thinking to themselves “Getting Joe to work a bit harder for a while will get this done faster, and be easier than reducing physical costs. Let's do that!” Without data to back up assertions that the goals imposed as a result are not achievable through reasonable effort, the programmer is left in the lurch – they have no power to change the situation, and thus often end up working long hours to meet the seemingly arbitrary goals.



The Problem
The primary problem with the “just work more” approach is obvious to anyone who has done creative work for any length of time – creative work is exhausting. Performing that kind of work for any length of time can feel very much like running a marathon, conflating the mental and physical worlds into a conglomeration of perceived Herculean effort. After X number of hours, with X being different for every individual, it is simply not possible to produce at the same quality level as it was possible before X hours had elapsed.



The practical upshot of this is that quality of output for creative endeavors decreases as the time spent on that task in a session increases. This is the central point that is missed when attempting to procure more effort out of employees who derive their primary work product from the expense of mental effort – longer hours means more bugs and issues, which means longer hours to fix them. It's an infinite loop of increasing required effort, which often ends up costing more to complete than if the employee had been allowed to go at their own pace.



Note that the video game industry is horrible in this regard. Long hours (60-80 hour weeks or more) are the norm in that industry as projects move towards the completed stage. Project managers in the game industry are often constrained by hard dates – such as the holiday season – that they have no way to negotiate around, and so when you cannot manipulate your end date the only thing remaining is to attempt to manipulate the amount of effort performed each week. In many cases, this results in months of extremely long hours, where programmers and artists are pushed relentlessly to meet deadlines that are forever seeming farther away. Additionally, the cumulative exhaustion of weeks of long hours seeps in, resulting in the remaining amount of work increasing despite the longer hours as bugs introduced into the system by a tired programmer required effort to fix.



Having spent time in the game industry, I wish I could say that the above was an apocryphal example, but examples abound throughout the industry, and things rarely seem to improve. The marketing and funding realities of the industry make this a problem that is tricky to solve. Some companies have large enough cash reserves that they can simply ignore the calendar and say “It'll be done when it's done,” but most others need the infusion of cash provided by a completed project to continue to do things like pay rent, electricity, and employees (often in that order).



Analyzing the issue
The game industry has a situation that arises from external constraints, but it also is a byproduct of creative work – there is no way to know how long a task is going to take before it is started. Sure, you can have a ballpark. For example, I've written two books so far, and each took about 30 days for the final draft. Given that, it's reasonable to assume that if I were to write a third book, it would also take about 30 days for the final draft.



Except when it doesn't.



This is the most frustrating thing about programming to non-programmers. Two tasks can seem extremely similar, but one may take five minutes while another would take a week. The reasons why are numerous – fractured code bases, incompatible patterns, and so on – but ultimately there is no obvious reason that a layman can point to and say “Oh, that's why that took so long!” This problem, when repeated multiple times over the course of a project, can sour the relationship between a project manager and a programmer, taking what was originally a productive and respectful interaction process and making it confrontational. It is usually not the result of any intentional slight by the programmer or intentional misinterpretation by the PM, but instead it's an issue of trust.



Every time a programmer claims that a task will take longer than originally expected, or that they have been delayed, the PM has no choice but to assume that the delays are intentional as they have no insight whatsoever into the development process. This may not be an overt assumption, but it certainly occurs on some subconscious level and factors into decision making and interactions. This injects stress and discord into the process, which builds up over the course of a project and results in negative perceptions. A project could be on the most efficient possible track to completion, making strides that no other team could possibly make with the same material, but once the perception is soured that effort will never be seen as sufficient.



Addressing the Problem
The problem, as we've already mentioned, is that the PM has no way to know when a given task will be completed. The frustrating thing is that oftentimes, neither does the programmer. Sure a programmer can make an educated guess, but there are always factors outside of their control that play into the end result. Thus, the first approach to addressing the problem is to rely solely upon the guesses of the programmer. The programmer takes a look at a task before he begins, and gives an estimate of the effort required to complete that task. As a project manager is concerned with dates and timelines, they often push for time-based estimates (“It will take four hours”), and many programmers agree to the system.



Problem Solved, Right?
Once a unit associated with time has been decided upon, everything very quickly goes to hell. If a task that should “only take four hours” suddenly takes 40, then there is a perceived deficit. A project manager looking at a project plan sees the extra 36 hours of effort and is forced to conclude that “The project is now 36 hours behind.” This amount grows or shrinks as effort continues, and the perception of the project manager changes each time. If a task takes less time than expected, then the programmer is “making good progress” and the project is “on track.” If a task takes more, then the programmer is “introducing delays” and the project is “delayed.”



Notice the attribution of effort here. This all ties back to the choice of time as a base unit. Once you are estimating in terms of hours, the natural human tendency is to start watching the clock. We've been trained to pay close attention to time periods – 8 hours of work, 1 hour for lunch, 30 minutes to commute, and so on – that we subconsciously monitor the amount of time taken. When something takes longer than stated, it is frustrating and unsettling. By tying the estimates of a programming task into this time-based system on a granular level, we introduce the same predilections and perceptions into the evaluation of progress. It removes the ability to look at a task objectively and say “we are making X amount of progress.” Instead, everything is cast in terms of time. Time elapsed, time remaining, time behind, and when looking at the time associated with a project subjective judgments are added into the project status. Choosing an estimate in terms of time has resulted in looking at the project at a micro level, removing the ability to see the forest through the trees. The subjective perceptions of “late” and “early” that have been pounded in to us from our youngest years come into play, and color our perception of the project's progress.



Aggregate Data
One of the reasons behind this is that it is far too easy to look at a project as a series of finite tasks. Once these tasks have a discrete time associated with them, it is far too easy to take each task as an element in isolation. Good project managers, though, recognize that each task is only part of a larger whole. They strive to look at a project in aggregate – that is, as a combination of all of the tasks. They see the four hour task taking 40 hours, but then realize that the project is still 25% completed with one quarter of the budgeted time elapsed.



Thus, we see the path to our solution. An individual four hour task may run long, and may result in a delay. But on average, four hour tasks will typically take four hours. Some may be shorter, some much longer, as that is the nature of creative endeavors. However, looked at over a span of time, it is possible to develop a translation. If the average of all “four hour” tasks is 4.5 hours, that gives us a reliable predictive measure. We now know that if the programmer estimates a task at 4 hours, it will take 4.5 hours on average. This allows us to adjust our estimates and determine where the project actually lies.



A Problem of Units
At its core, with this translation table we've improved upon the primary concern – developing accurate estimates of project effort. However, we haven't addressed the soft factors of the issue. By associating progress with hours of effort, we are still tapping into that primal clock-watching urge, and thus still adding in the subjective judgements that go along with it. This is the problem with any time-based estimate system – it is tied to a known and familiar system, and even though it may operate on a varying scale, there is a subconscious perception of variation that permeates the project, regardless of the intentions. Because “hours” was chosen as the unit, this is tied back to real world elapsed time, and we end up with a slightly improved version of the same problem.

As it is, we now actually have the seeds of a potential solution. We've identified that looking at effort in aggregate provides us with more accurate estimates of project completion. We've also identified that expressing estimates in terms of time introduces needless subjective judgments into the process. We finally know that despite our best efforts, we do eventually have deadlines to meet – we cannot continue to just work forever.



A Different Approach
What we are really doing when estimating a task is building a view of the entire project. A project is the sum of its parts, and having broken a project down into constituent tasks it thus follows that a project's total effort is the sum of the efforts of all of its tasks (it is more complicated than that, but usually the point holds well enough). So by adding up all of the estimates associated with a project, we can determine a total estimate for the entire project. In a time-based system, this is expressed in terms of hours and days.



However, at this point we are working simply with numbers. We've established that due to the nature of creative work estimates are by definition non-concrete. So instead, we apply analytics to find out what “four hours” actually means in terms of calendar time. At this point, we're simply doing number analysis – averaging numbers together to produce a measure in terms of time. Since we have the same units on both sides of the equation, it follows that when these values on either side of the equation differ, then one of the choices of units is thus incorrect. It should be obvious that 4.5 hours does not equal 4 hours, so one of those figures must not really be “hours” as we know them.



Let's instead refer to the right side of that equation as points. Instead of 4.5 hours = 4 hours, we have 4.5 hours = 4 points. By removing the hours from the right hand side of that equation, we've suddenly addressed the subjective issue with our measurement system so far – we took away the time base in order to find the actual value, and as a result hit upon a number system that allows us to assign arbitrary values to tasks that develop meaning as a project progresses.



Building upon this gives us a more reliable means of estimation. Instead of assigning an hour estimate to each task, we assign a point value. The point value is indicative of the general effort required to complete the task, just as the estimate in hours was, but by removing the time value we remove subjective expectations. We assign higher numbers to challenging tasks, and lower numbers to easier tasks, knowing that on average the effort expended for each classification of task will average out to the same amount of effort, even if one specific task takes longer than another.



Tying it All Together
As we noted earlier, deadlines are inescapable. We cannot simply work forever. Thus, status reporting becomes a matter of determining how much longer the project will take to complete. We have a method of assigning difficulty to our tasks by giving them point values, and the point values are arbitrary units and thus remove all subjective judgments and perceptions. So our task at this point is to tie these points back into time periods.



This is actually what we were hoping to accomplish with our original estimation scheme – determine the amount of time it will take to complete a project. We simply hit upon the wrong units, and thus added a lot of extra cruft to our project plan. By trying to tie points back to hours, we become too granular – the points completed per hour varies far too much from task to task to give a worthwhile measurement. So the only option is to expand our averaging time window. This becomes an arbitrary choice, but I've seen the most success with week-long and two-week-long periods, known in the agile methodology as “Sprints”. You add up the points completed during each sprint, then keep a running average of the efforts completed. In doing so you tie the point values back into time values while also accounting for the aggregate variance of the data. For example – if I have a project that is has 170 units remaining, and I have been averaging 17 points completed per one-week sprint, I can say with a fair degree of confidence that there are 10 weeks remaining in the project.



This 17 number – the average point value completed during a sprint – is known as “velocity.” A project's velocity is used to relate the overall project duration back to the time required to complete it. By increasing the unit of measurement and divesting our estimates from subjective measures, we've introduced a more reliable system of estimation – simply divide points remaining by velocity. What this does is twofold. First, it gives you a quick and dirty estimate of the time remaining in a project that, over several weeks, tends to become very accurate as the amount of data to measure increases.



The second thing that it gives you is the ability to say no. Take, for example, a project that is starting to trend towards long hours. If you archive your data properly, you can point to previous examples where such an approach only made matters worse. Instead of “well, that will be pretty hard on the team,” you can instead say “We tried this before, and added 50 points of work while only achieving 47 from the increase in face time. How do you think it will be different this time around?” By looking at the point value of each task to be completed, some negotiation can take place and instead of giving subjective measurements and impressions, you can provide specific feedback with actionable data. Of course in some cases this won't do a damn bit of good, as the project deadlines may be particularly unforgiving, but in aggregate this approach should produce an environment that is collaborative and built upon a reliable history, as opposed to a relationship founded on distrust and fostered in a confrontational environment.



What does this mean for writing?

Ultimately, this method obtained from programming can be applied to any creative endeavor that can be assigned a subjective judgment of “easy” or “hard.” Simply break a project down into constituent tasks, then assign a point value to each task representing its difficulty. Of course after a certain point the tasks become repetitive – how many ways can “Write this scene!” be phrased, after all? However, one thing I have found when applying this approach to my own efforts is in support of my previous post on metrics – once you start putting your mind to measuring your progress in a more objective manner, you are able to get a better feel for how things are actually going with the project. While you may or may not have deadlines to work towards, and you may or may not have people waiting on your progress, the concrete proof of your progress is often enough to help you break through any bumps on the road, and stave off periods of despair over how long a given project is taking.

Video Game Review - Mass Effect 3

I recently finished Mass Effect 3, and I am left conflicted. On the one hand I've greatly enjoyed the series, and feel that the content team has created a very rich universe with a detailed and interesting history to learn about. That being said, I had a few issues that I wanted to get up on my soapbox about:

• First, a pet peeve: Decimate (v) - to reduce by ONE TENTH. See the first two definitions here. I get that people are trying to change the meaning of the word because they don't understand it, but this shit should have been caught during the "peer review" process Bioware is so proud of (the same one that failed them on the ending, according to reports). Just because a word sounds cool, that doesn't mean it's OK to completely bastardize its definition, regardless of what the unwashed masses have come to believe that the word means! Looking for a cool synonym for destroyed? Why not try demolished, or annihilated, or eliminated, or ravaged., or any of the actual synonyms for destroyed?
• That being said, there were some truly moving moments in the game's story. There were also some areas in which the dialog sounded like it was written by a 14-year-old Call of Duty fanboy. The sad part is that the quality of the rest of the writing makes the inferior text stand out, tearing at my eyes with horrible use of invective. I swear at one point I saw Shepard say something along the lines of "Git 'er done!" I know the game was delayed for six months, but that doesn't mean you give up on quality control completely!
• Finally, another pet peeve of mine: if I'm playing a game with achievements, why not give me those awards when I actually, I don't know... achieve something?! Kill 100 enemies with Incinerate? Cool, that's actually something worth striving for. However, doling out gamer points like breadcrumbs as you advance through the story doing things you have to do anyway only calls attention to the fact that your game's gameplay sequences serve as nothing more than interstitials between cutscenes. The time spent designing these bogus achievements (things along the lines of "Completed story mission 1, completed story mission 2, completed...") could have been used to design achievements that actually add something to the game. It would have given me something to do while muddling through your combat system (which made the mistake of trying to be one of the standard brown shooters that are so popular with the kiddies lately), which could only have added to the gameplay. Instead, as a result, the entire experience just comes out flat.
• Along those same lines - maybe it's the way I play, but I only found the gameplay exciting once - right at the end of the game. All the rest of the time was "hide, shoot, hide, shoot, move, shoot, yawm." That being said, I do like the new movement options as opposed to prior games in the series, and I feel that the combat system has come a long way.

Some of these are kinda nit-picky. I'll admit to that. Like I said - I did overall enjoy the game, but it was definitely not gaming perfection. What are your thoughts?

Ranting - On form factors

I'm a big fan of e-readers. And by e-reader, I don't mean your Kindle Fire or Nook Color - I'm talking about actual, e-ink-based readers whose sole purpose is to replace the hundreds of pounds of books on my shelf with a single device. Things like the Sony Reader (my first e-reader was a PRS 350), or the Kindle Paperwhite (my newest reader, a gift from my lovely wife and son). These devices are intended to display text, and that is pretty much their only goal. In my opinion, they meet this goal in an absolutely stellar way. I can carry enough entertainment around in my pocket for years worth of idle-time, and the design goals of the e-ink display make sure that I don't end up with a headache for my troubles.

That's why I'm troubled when I see so many people opting for tablets as their reader of choice. With e-reader sales declining, far too many people are choosing an inefficient and incorrect medium to consume their text-based media. For me, the advantages of e-ink over a tablet are as follows:

• Reduced eye strain. An LCD is basically a bright light shining at your face. An e-ink reader relies upon ambient light, much like a piece of paper.
• Increased battery life. An LCD takes a hell of a lot of power compared to e-ink, and a lot of it is due to the frequency of the screen's refresh. The more often a screen has to change, the more power it consumes. E-ink displays change only when you switch pages, resulting in batteries that last for weeks. LCD screens change dozens of times a second, resulting in much greater battery usage.
• Reduced glare and contrast ratio improvements. LCD is a light-based medium, emitting light in order to generate the images we see. This means that if you shine a bright light back at it (say, something like the sun) then the image becomes much more challenging to discern. This is not a problem with e-ink readers, as they rely upon the reflection of light off of their surface - just like a piece of paper. This also results in a far higher contrast ratio - the difference between dark and light areas on the screen. This higher contrast ratio on e-readers allows the eye to do the same tasks with less energy, resulting in the reduced eye strain mentioned above.

The problem I see is that people want one device that does everything they could ask for - a Swiss army knife of entertainment. This results in millions of misguided individuals using their LCD tablets as a repository for their books and reading material, resulting in headaches and unnecessary power usage.

The issue is all about form factor. The central fallacy that lies behind these choices of gadget is that one form factor can fit every need. You see this confusion all over the place. One example is Microsoft's Windows 8 - a desktop OS heavily designed for touch. Touch interfaces can be excellent for some uses, but with the popularity of the iPad and smartphones far too many people think that a touch interface is the panacea for all computer applications. Touch is imprecise, it lacks sufficient tactile feedback for button presses and such, and requires a flexible display to compensate for the lack of resolution available. Furthermore, it often requires additional software running on a device to handle the input. Instead of a device driver generating interrupt requests, you're filtering the input through an application subject to the whims of an OS.

Human experience, at least in America, seems to be far too focused on fads. A new technology, or product, appears and instantly trumps anything even remotely related to it, regardless of the superior technology. VHS over Betamax. Touch-screen keyboards over physical keys. I suspect that tablet computers are quickly doing this to e-readers, and it makes me sad as both a tech guy and as a reader. I'm not looking forward to the forthcoming headaches once all devices carry a brightly-glowing LCD. I stare at a computer monitor too long as it is.

On Workshops and Wetware

This past Friday I attended a workshop presented by April Eberhardt and Susan Bearman entitled “Pathways to Publication.”  It was a very interesting workshop, presenting the “current” view of the ever-changing publication landscape as lived by two professionals. There was quite an eclectic group gathered in the conference room of Chicago’s University Center, representing essentially all stages of the production process and most of the phases of a literary career.

I have to say that while I had discovered a lot of the subject material on my own, I still found both the presentation and discussion useful. If you have an opportunity to attend a talk by one of these two women, I highly recommend the opportunity – they are both enthusiastic about their subjects, and energetic and engaging in delivery. Susan is an author as well as a marketing communications specialist, and April an agent with a list of clients at varying levels of the writing process.

My Primary Takeaways 

My primary takeaways from the conference were, in many ways, things I already knew. I am not a published author, nor am I particularly experienced at writing salable fiction, but I am (if I can toot my own horn here) excellent at researching topics and distilling the knowledge down to the salient points. Along these lines, I have done a lot of reading about writing and publication and have read points on all sides of the issue.

I tend not to write about publication and my opinions on the “right” or “wrong” way for two fairly simple reasons:

1. I am not yet published
2. I’d rather write than write about writing

There are hundreds of thousands of words out there about the various ways to get published, and they all boil down into the following categories:

• Individuals who firmly believe that traditional publishing is the only way forward, and any who disagree are heartless blasphemers who will never shake the negative stigma of the self publisher
• Individuals who firmly believe that indie publishing is the only way forward, and any who disagree are heartless blasphemers who will never earn even a tenth of their self-published peers
• Individuals (and, most likely, the truth) who lie somewhere in between these two extremes

I use religious terminology in the above because in many cases that is the style that is taken by these blog posts and various discussions – absolutes thrown out with aplomb by individuals wholly dedicated to their particular cause, and god help the poor sap who dares to express a contrary opinion.

These people are not to be taken seriously. At least, not by me. They are informed by their own success, and thus may be biased towards what has worked for them. That is not to say that they don’t have any useful information, but arguments and opinions should always be taken with a grain of salt.

And ultimately, that was my key takeaway from the workshop. There is no right path. There are different paths, suited to different individuals, with different goals. Some people may want to make a living from their writing. An exceedingly small percentage of those people may achieve that goal. Some may only want the cachet of the title “Published Author,” and a small percentage may achieve that goal. Note the repetition of the phrase “small percentage” – the only true reality is that thousands and thousands and thousands of books are produced each year, and only a small number of those obtain publishing deals, let alone any kind of cultural relevance.

So in the end, this is what I gathered (right or wrong) from my time at the workshop:

• Know your goal – am I looking to hit it big, or do I want the respect of a published writer, or do I simply want to create something that will live after I have died?
• Plan your path forwards – what are measurable steps I can take towards that goal? How much will the options available to me cost me in time and/or money? How do I know when my goal has been achieved?
• Do not dismiss any options until you are absolutely sure that they are not viable – the numbers game is already heavily weighted against me, would dismissing an entire market vertical help?
• Finally, make your own conclusions – the only person with all the data on my situation is me, not an opinionated blogger or a pretentious literary critic, so why should their passion inform my decision if their reasons aren’t applicable?

The Fringe-Benefits, and Why I Will Never Be a Politician 

As any experienced conference attendee will tell you, the knowledge presented is only half of the reason for attending the event. If the knowledge was the only important thing, we could just download the PDF and go on our merry way as newly-enlightened writers. The other half of every conference, expo, convention, party, box social, et cetera, is the social aspect.

When I was a young college student getting ready to make my way in the “professional” world, I had a strong aversion to networking. I was earning a degree in Computer Science, which is based heavily in mathematics. In math, you are right or you are wrong, and there is no real room for in-betweens – that’s what statistics and probability are for. As I of course knew everything possible to know about the world at that age, I firmly asserted that I had no need of networking with potential colleagues. That kind of glad-handing may be necessary for business people and politicians, but I was a programmer. I just had to be the best programmer, and bam – job time for me. I envisioned myself living in a world of true merit, where the most talented were rewarded and the bull-shitters would be exposed and shamed.

We were all so naïve, once.

Some people say that money makes the world go ‘round. However, behind that money is a directing hand. This essentially means that the hand driving the money is just as important to the angular momentum of our planet as the money is itself. Thus, there are two paths to having an effect on the world:

• Have lots and lots of money at your disposal
• Connect with enough people with similar goals and work together to achieve what you need

Now this is a simplistic view of the world dedicated to show one fact: the only people who can afford to ignore the social side of the world are people with the fiscal resources to do so. Want to advance in your career? See and be seen. It is not enough to be damn good at what you do – that will get you local notoriety and aplomb, sure. If you want to make it big, though, you have to advertise. Be your own salesperson. Know your upsides and down, and emphasize the up while diminishing the down. Talk to people. Connect. Network. Make friends and find colleagues.

And this is where I fail. I like to describe myself as a quiet person. I have few social skills of which I am aware, and my sarcastic and self-deprecating nature can be off-putting to some people. On top of this, I have… while not exactly a fear of social interaction, I certainly have some sort of mental block against it. I rarely feel confident when introducing myself, and I’m by no means a people-person. To put it simply, I’m not an aloof ass – I’m just shy.

This is an issue that has plagued me for a while, and this past Friday it more or less reached a tipping point. I’m not great at networking (I even forgot to have new business cards made, as I am no longer a graduate student in computer graphics), and I tend to hang back by myself in a crowd. Here I am with a room full of people who can prove great people to know and potentially build a long-term correspondence with, and outside of a few smiles I only really talk to the people who approach me directly. A fantastic opportunity somewhat wasted.

In computer programming we see three phases of every solution: the hardware, with all the lights and sounds, that makes the program run; the software, with the thousands of lines of code, that makes the computer do what you want; and the wetware, with the hair and the appetites and the sporting team affiliations, that directs the entire process. Like far too many computer programmers over time I have focused a large portion of my attention on the two former, and not nearly enough on the latter.

And that’s probably my ultimate take-away from the workshop on Friday. Somehow I need to learn to be a people person, because even though I might be able to tell n interesting tale, nobody will read it if they don’t know who I am.

Long ramble, complete. Shutting down for now

On long-running series

With the release of A Memory of Light, Robert Jordan's (and, ultimately, Brandon Sanderson's) Wheel of Time series drew to a long-awaited close. Having started reading the series back in 1994, I was understandably excited about how the series would wrap up. So excited, in fact, that I set aside several months of time to completely re-read the series from start to finish.

Three months later, I was ready to throw all of the books away.

I had grown incredibly sick of his portrayal of women. I got tired of the viewpoint switching just when things got interesting. If Nynaeve gripped her braid in one more white-knuckled fist, I was going to write a fanfic with the express purpose of having very painful and degrading things happen to her. The writing style, which hadn't bothered me too severely before, just ended up grating on me.

I've noticed this a couple of other times when I've opted to plow through a long series. The Robotech novelizations were interesting, but after twenty books of very light reading I ended up less than thrilled with them. And David Weber's Honor Harrington series... I never even got all the way through that one. After reading an entire paragraph devoted to the nuances in emotion that the titular character gleaned from a single look, I was ready to throw my e-reader at the wall. "He had a stern look in his eye, but beneath that she could see the glimmer of mischievousness, buoyed by their past relationship. But she could also see the sorrow in them over her recent loss, and felt gratitude at the sincere regret she read in his eyes as well." I exaggerate, but only a bit. After the umpteenth description along those lines, I simply could not continue - that series of novels stands today as one of the only ones that I will not finish after having started reading them.

The issue is not with the writers themselves - the respective authors are much more accomplished than I, and have earned massive respect for their accomplishments. Nor do I think I could do a better job - that's not the point of this post. Instead, what this tells me is that a long-running series can simultaneously be an author's most beneficial characteristic as well as one of their most detrimental assets.

The key is in the subconscious patterns that evolve from works of any particular length. While the inter-gender disputes in the Wheel of Time were at first a minor annoyance behind a novel fantasy tale, over the course of the series they became much more pronounced.  Honor Harrington's empathy was a character trait that had evolved from events in the series, but after a while it came to dominate her every interaction with other characters. Terry Goodkind's objectivist philosophy grew from the ideas behind the actions of Zed and Richard into an entire guiding philosophy of the series - something that was most certainly not evident before the publication of Blood of the Fold.

I could go on, but I think my point is made. After a certain number of words, these traits become more pronounced, the characteristics take up more and more page space. Why does this happen? Is it simply something with which the author can fill page space in a familiar manner? Or is it a symptom of how the books are consumed? For example - if I read one Wheel of Time book a month, instead of one after the other, would I have been as emotionally invested in hating the gender portrayals at the end?

Maybe I'm just hard to please. And if so, what does that mean for me as a writer? Is there some day where a reader of my work will send me an angry letter detailing how I am everything that is wrong with humanity? Is that, necessarily speaking, a bad thing?

Maybe this is just a reason to avoid long-running series in general. Either writing or reading them.

The contest has come to an end, but the journey is just beginning

Today, the Amazon Breakthrough Novel Award announced the 25 semifinalists. Unfortunately, yours truly was not among them. I wish the winners the best of luck - I am sure they are all fine examples of the art, and will see great success.

As for me, I learned a lot during this process. I'm not really upset or irritated that I wasn't selected - on the contrary, I was honestly surprised to have made it as far as I did. Being selected as one of the top 500 entries out of 10,000 submitted is an honor I'm still not certain I'm worthy of, and the forthcoming Publisher's Weekly review - whether it is positive or negative - is simply icing on the cake. The fact that this is my first novel - indeed the first work I've written that is longer than 15,000 words - is just something I am immensely proud of.

Sorry if the previous paragraph sounds like bragging - it's honestly not intended that way. I just never expected to make it as far as I did.

Now, where do we go from here?

Let's talk about a couple things: the things I learned from this contest, and what I plan to do with it.

On Reviews

Firstly, I'll probably keep obsessively refreshing my reviews page in createspace. While my position as a quarter-finalist warrants me a Publisher's Weekly review, it hasn't actually been posted yet. On top of that is the fact that the review is not required to be a good review. There's no such thing as bad press in many cases, but I can't say I'm looking forward to being excoriated by one of the gatekeepers of the publishing world.

That being said, I also earned two reviews from Amazon "expert" reviewers. I mean no disrespect by the quotation marks - I've simply heard them referred to by a number of terms, and I'm unsure what exactly brought them to the rank of expert. There were two, both of which contributed to the decision to move my book onward to the quarterfinals. Both reviewers praised my writing style - one called my writing style "gripping," while the other described the writing as "strong". At that point the two reviews diverged. One claimed that my ability to seamlessly move the viewpoint between the character's internal dialog and the actual events was excellent, while the other said that they found it jarring in places. One said that he disliked my main character based on his word choices, while the other described my main character as "dynamic."

Probably the most troubling part of one review was when one reviewer stated their "disinterest in the science fiction portion of the novel." I have yet to determine if this is due to the reviewer's dislike of science fiction as a genre, or whether I just did not present enough science fiction elements fast enough. If it's the former (which I am led to believe based upon my reading of how the ABNA process works), then I question the system Amazon has put into place for this contest. Maybe it was just a fluke, but it's a disconcerting one.

In either case, I wish I had the opportunity to thank both of my reviewers. I gleaned a lot from the brief reviews they left, and I will apply it to my future writing.

Moving Forward

My conclusion after this contest is that, barring a horrid review from Publisher's Weekly, my book just might be ready for marketing. I've been pondering what to do with my writing for a while now. I have a stable day job that pays very well, so I'm not looking to make a living on my writing just yet. Granted that's one of the end goals - to do what I want, essentially, for a living - but the process I plan on taking isn't necessarily driven by publication as any kind of hard benchmark. If my novel isn't worthy of the world, I have no problem writing another (and another and another) until I get it right. Or until I've saturated the market.

I already have a sequel written for the novel I submitted to this contest - Majestic - as I feel that I had places to take the story that the first novel didn't adequately explore. This novel hasn't been edited (or even read yet), but I already have ideas for others in a different series. Aside from that I've been dabbling in short story and scene writing (which you can see on this blog), just generally building my skills. I look at writing like programming - you can spend hours and days and years tightening the project down, but after a while you just need to get it out the door and move on to something else.

On Self Publishing versus Traditional Publishing

There is endless debate online about self publishing versus traditional publishing, and it ultimately boils down to this: If you self publish, your royalties for each book you sell will be higher on average, but the perception of your quality will be lower, as will your out-of-pocket expense. If you traditionally publish, you get the cachet of the the published author and better publicity, but you run the risk of not earning as much in royalties.

Personally, I see merits in both approaches. Self-publishing gets you feedback and income now, but removes the barriers to entry. There are a lot of bad books out there - poorly edited, poorly thought out, poorly formatted - that people throw up just to be able to say "I'm an author, you can buy my book here." These books bring down the perception of self publishing as a whole, and actively harm the community. Honestly, these books are the single biggest argument against self-publishing for me. Although I need to be honest and admit to myself that there is a good chance that my book also falls into that category.

Traditional publishing, on the other hand, has the gatekeepers. You need to find an agent, who will request edits, who will sell to a publisher, who will request edits, who will print the book and sell to bookstores. Then you need to participate in marketing with variable support from your publisher, who may be loathe to take risks on a neophyte author with no proven sales record. In the end you have your book in a bookstore, but you're taking on a lot of the process than authors have traditionally done in the past.

So which is important to me - getting it out there, or getting the respect?

My Approach

My primary consideration is that  I recognize that I'm trying to build a brand. After a while, if no one has heard of me, no one will buy my books if I ever end up published. So here's my naive plan to get the ball rolling:

1. Start the query process. Get my book out and in front of agents, and see if I can get one to bite.
2. After twelve months, if I have not received any substantive indicator of progress, pursue the self-publishing route.

This is the plan I start today, with my first official query later this evening. I've a number of reasons for choosing this approach, but they can be summarized as follows:

• Some writers see romanticism in amassing years worth of rejections before finally getting that one "yes" they need. I don't really see the point in this, myself.
• In some ways, being proliferate can trump being unknown. Having things out and selling builds my name, and builds evidence behind me that I have something of worth to offer the reading world.
• Ultimately, I'm writing for myself. If other people think my books are publication worthy, that's great! However, I don't plan on truly defining my success as a writer on my publication status. As such, I'm not really in the mood to beat my head against a brick wall, hoping I break my way through after enough strikes.

So what do you think? Am I making a good choice, or taking the first steps down the path of folly?