Design Principle: Favor Composition Over Inheritance

Here at HauteLook, we’re almost constantly interviewing engineers. Not because we have high turnover, but because we’re always growing the team and it’s difficult to find engineers that have the skill level required to join our team. Part of our interview includes a “virtual whiteboard” using a Google doc. And part of that virtual whiteboard exercise includes representing a complex object, showing its properties and methods. We don’t ask for (or care about) valid UML; all that we care about is seeing that the candidate understands object-oriented design (OOD).

Many of our candidates do a wonderful job of constructing a complex hierarchy of inheritance. This gets the job done, but it doesn’t really allow a system to be extensible. When you overuse inheritance, you are basically designing according to assumptions which may seem true at the time, but which will likely not be true as the application is required to change. A key design principle is Encapsulate What Varies. The design principle covered in the post you’re currently reading, Favor Composition Over Inheritance, is one way of following the Encapsulate What Varies principle.

The overuse of inheritance in OOD is common, and is understandable. Many code examples in textbooks show inheritance as the way to make code reusable. And a design that utilizes a nice inheritance hierarchy does appear to make good sense. However, when you need to make a system do something that you hadn’t originally designed it for, the benefits you thought you were getting from an inheritance hierarchy suddenly become the sole reason to refactor your code. When subclasses inherit multiple behaviors from a parent class, they are locked into having those behaviors. The solution is simple, right? Just override those methods and implement the desired behavior for that subclass. But what if another subclass of the same parent has the same issue? Now, if you override the behavior in that subclass too, you have duplicated code.

Let me provide an example. (Note that I’m not going to design classes as we ask for in our whiteboard exercise. Instead, I’ll demonstrate this principle with code.) Let’s say that we want to design a class that plays music. The two concrete examples that we are required to implement are a record player (to play all the awesome 70’s music) and an 8-track player (just because). So we design a base AbstractPlayer class. It might look something like this:

abstract class AbstractPlayer
{
    public function play()
    {
        echo "I'm playing music through my speakers!";
    }

    public function stop()
    {
        echo "I'm not playing music anymore.";
    }
}

Looks simple enough, right? And our RecordPlayer and EightTrackPlayer classes would inherit from AbstractPlayer and would therefore inherit the play() and stop() methods:

class RecordPlayer extends AbstractPlayer
{

}

class EightTrackPlayer extends AbstractPlayer
{

}

Here’s some client code using the RecordPlayer:

$record_player = new RecordPlayer;
$record_player->play(); // echoes "I'm playing music through my speakers!"

Awesome! So now, some time goes by, and we get a request to implement a portable cassette player. It would probably be just like the above two concrete classes, but it would need to be able to play through headphones instead of speakers. Here’s what it might look like:

class PortableCassettePlayer extends AbstractPlayer
{
    public function play()
    {
        echo "I'm playing music through headphones!";
    }
}

Makes sense, right? Our PortableCassettePlayer inherits from AbstractPlayer and tweaks the play() method a little. Now, fast-forward a couple of decades, and we need to implement an Mp3Player class.

class Mp3Player extends AbstractPlayer
{
    public function play()
    {
        echo "I'm playing music through headphones!";
    }
}

Notice that we essentially copied the play() method from the PortableCassettePlayer class. We couldn’t just use the play() method from the base class because the Mp3Player can have headphones plugged in. Maybe we could inherit from the PortableCassettePlayer class, but then we’d inherit a ton of other functionality, such as ejecting a cassette tape, which doesn’t make sense for an MP3 player. The problem here is that inheritance has locked us into certain assumptions, and the design that was intended to encourage code reuse actually causes code duplication.

Instead of using inheritance to reuse functionality, let’s look at a design that uses composition. First, we’ll encapsulate the play() behavior:

interface PlayBehaviorInterface
{
    public function play();
}

class PlayBehaviorSpeakers implements PlayBehaviorInterface
{
    public function play()
    {
        echo "I'm playing music through my speakers!";
    }
}

class PlayBehaviorHeadphones implements PlayBehaviorInterface
{
    public function play()
    {
        echo "I'm playing music through headphones!";
    }
}

Now the play functionality is encapsulated into classes. Maybe you’re thinking, “aren’t classes supposed to represent things?” Well, in this case, the thing they’re representing is a behavior.

Let’s see how the Player classes can now use these new behavior classes.

abstract class AbstractPlayer
{
    protected $play_behavior;

    abstract protected function _createPlayBehavior();

    public function __construct()
    {
        $this->setPlayBehavior($this->_createPlayBehavior());
    }

    public function play()
    {
        $this->play_behavior->play();
    }

    public function setPlayBehavior(PlayBehaviorInterface $play_behavior)
    {
        $this->play_behavior = $play_behavior;
    }

    public function stop()
    {
        echo "I'm not playing music anymore.";
    }
}

Notice that the AbstractPlayer has a constructor which sets the default player behavior. It also has a new setPlayBehavior() method. This allows to set whatever play behavior we want at runtime. It also requires implementation of a method called _createPlayBehavior(), so concrete classes are forced to provide their default player behavior. (see my post on Removing Dependencies With Factory Method.)

Let’s create our Mp3Player class:

class Mp3Player extends AbstractPlayer
{
    protected function _createPlayBehavior()
    {
        return new PlayBehaviorHeadphones;
    }
}

The Mp3Player class sets an instance of PlayBehaviorHeadphones as its play behavior. Pretty cool, right? So imagine that a few more years pass, and now our Mp3Player needs to also support playing through a Bluetooth connection. What do we need to change in our Mp3Player class? Think about it for a second. The answer: absolutely nothing! We simply write a new class to encapsulate this new behavior:

class PlayBehaviorBluetooth implements PlayBehaviorInterface
{
    public function play()
    {
        echo "I'm playing music wirelessly through Bluetooth!";
    }
}

This new behavior class can be plugged into the Mp3Player class at runtime if the player is meant to support this functionality. The client code that instantiates the Mp3Player class handles this:

$mp3_player = new Mp3Player;
$mp3_player->play(); //echoes "I'm playing music through headphones!"
$mp3_player->setPlayBehavior(new PlayBehaviorBluetooth);
$mp3_player->play(); //echoes "I'm playing music wirelessly through Bluetooth!"

Beautiful, right? Encapsulating behaviors into separate classes gives us true extensibility. In fact, design principles such as Favor Composition Over Inheritance are the reason that we have design patterns. The main pattern used in the above examples is Strategy Pattern. Some other patterns that demonstrate this principle are Observer, Decorator, and State.

Oh, and one more thing to mention: the fact that we were able to drop in new functionality without changing existing code exemplifies another awesome design principle, the Open-Closed Principle. This principle states that classes should be closed to modification but open to extension.

As you think about how to structure your code, I encourage you to keep this principle in mind, and see how it can improve your design. You’ll thank yourself when you have to go back and add new functionality.

iOS Hackathon

The challenge was to take a group of non-iOS developers, and teach them enough about Objective-C and iPhone development that they could make their own applications, and do this all within a 2 week timeframe. In retrospect, this was an amazing task to even attempt, and I was glad to be part of it. Here are my thoughts on the process:

Coming from PHP, Objective-C is a very strange looking language. The method signatures alone took me a few days to get used to, however now that I understand them they actually make a lot of sense. The real difficulty for me was to wrap my head around the following concepts:

Pointers
Pointers are just annoying, I understand how to use them but still it seems like something that could be easily hidden by the language. Also I don’t see any benefit by using them.

Everything is an object
PHP is an object oriented language, or it can be if you want it to be, but in Objective-C Everything is an object. You almost never use the native C datatypes, those are all objects. This means that you have to properly instantiate these objects by allocating memory to them and initializing them. My complaint here is that it adds a lot to the code even for doing something basic.

Strong typing
I understand the concept that strong typed languages force you to be explicit with your intent with regards to what parameters are passed too, and returned from, methods; and that this in the long run may make it easier to maintain the code, however it also makes it much more difficult to do simple things (like parsing a JSON string). Also, I have rarely seen an example where not knowing the exact type of a variable in PHP has caused a major problem. I’m not saying this doesn’t happen, but it seems to me that the advantage of week typing far outweighs the added complexity.

Mutable vs Immutable objects
My first reaction to this was “Why not make all objects Mutable?” of course this is coming from a background of PHP which does not have multiple threads. When you take concurrency into account Immutable objects make a lot of sense because they make it much less likely that you will run into a race condition. It makes you wonder why they didn’t introduce this same concept in JavaScript.

The xCode nib/xib graphical editor
When I code, I expect to see well… code you know? It’s very disconcerting for me to be taken to a screen where I must click and drag something and it does something else to some piece of code that I can’t actually see anywhere. It reminds me of working with the WYSIWYG on Dreamweaver and I don’t know why but it just infuriates me. You see, the nice thing about working with code alone is that I don’t have to guess where the problem is. If there is a problem then it must be right in front of my face, in the code; there is a misspelling or a colon where there should be a semicolon or something of that nature, but when I see a GUI, I have no idea where to look. How am I supposed to know that if you right click over this button and select this dropdown and expand this window you will get to see where the bug is?

So after a very short dive into this area of development my opinion on Objective-C and xCode is that I’m confident that the stricter rules of the language will in fact produce better, more manageable, more extensible code, but still I ask, is it all worth the trouble? Sure, if you are building an app for a well-established company and you know that this app is going to be around for many years it is definitely worth it, but isn’t it the case that 90% of all applications made will never become popular enough for anyone to want to use them even a few years after their initial release?

It seems to me that in most cases, you would be better off using some cross-platform solution like Corona or Titanium to build the first version of your app and save yourself lots of money/time and in the event that your app is actually a success only then do you re-write it using the actual language and proper tools.

I see Objective-C vs PHP much in the same way that I see Mac vs PC. Yes, the Mac is better built and will last longer than the PC but does it matter. I tend to want to replace my machine way before it comes to the point of physically failing so who cares what its ultimate lifespan is. But I concede that I don’t really know what is better, I have always opted for the cheaper/quicker solution. This, however is something I intend to change. Because of the hackathon I have decided to purchase my very first Mac ever and actually learn the language even more. Who knows, maybe in a year I will have a completely different perspective.

Magic Product Picker – iOS Hackathon

Last week we finished up our iOS Hackathon projects. I was very excited to participate in this since I had already started to dip into the world of Objective-C and was looking forward to dedicating a week towards educating myself more and having another week to actually make something. First week was training for us and for the first couple days we watched video tutorials as a group. For the rest of the week we did some hands on tutorials on our own. Doing this built a decent foundation for all of us who wanted to participate in the hackathon.

On Friday we had a meeting to talk about what our projects were going to be for the following week. The challenge was to have a Minimal Viable Product by Wednesday then for the rest of the week we would clean up the app and add any little extras. My idea was to create an app that when you shook the phone it would randomly show a product that was currently selling on the HauteLook site. I have always liked when websites or apps had a feature to randomly find items for you, it is a fun way to discover new products.

On Monday we were off, all of us quietly hacking away, jumping hurdles, trying to get our projects up and running. For me the first order of business was to grab the catalog of products from each of the events that were currently live. This was simple enough and once I did get all the data I stored it in an NSDictionary, this made it easy to pick out a random entry. By the end of the day I was able to shake the phone and it would show the title of a random product on the screen. Now I was coming up with a whole bunch of ideas for other things I wanted to implement that would either make it a much more fun experience for the user. Here is the list I came up with:

  1. The ability to filter random pick by category.
  2. A button under the product that let you Like/Dislike the product. The idea being it could be saved and, sometime in the future, it could be used in an algorithm to make random picks better aimed at the users preferences.
  3. An animation that would happen while it was magically choosing the perfect item for you.
  4. A detail page for the item chosen.

I figured I would not get all of these done but I could get a couple of them in place, I mean it was the first day and I already had the core feature working.

On Tuesday morning I was playing with the app and decided that I did not like how long it took for the app to launch. The delay was due to the fact that I was grabbing all live events then looping through each of them and making a separate call to get the product catalog for each event based on each event. I fixed this by changing the code to grab only the events on startup and leave the calls to get the product catalogs to only happen once the user decided to shake the phone. It would take a little longer for the random item to show up but this could easily be masked with a clever animation, something I had already thought of implementing anyway. With my performance issue solved I started to look at my proposed list of features and thinking about how I would get them done. The filter was the most important one, I knew I had to have that and the second on my list was the "Like/Dislike" feature. I wanted the Like/Dislike to be saved to the device so when the user came back and used the app later it could, at the very least, prevent the disliked items from showing up again. Seemed like using Core Data was the way to go so I popped open google and started to research it. I also figured I could store the events and products in Core Data, that way we would not have to fetch all new data every time the app was opened.

After reading a couple tutorials I started to refactor my project and it was not long before I was able to successfully save my first entity. But I did not like the idea of all the initialization code for Core Data being in the AppDelegate, which is where all the examples I saw were putting it. I needed a wrapper to contain all of this boilerplate code. This is where I should have stopped to think about how long this would take, while normally it is a good thing to take the time to make sure you have clean reusable code this was a "hackathon", hack is right there in the name and this code was is not supposed to have a long lifespan. But my inner "Uncle Bob" took over and clean code became the focus. I spent the next day or so experimenting and started with more refactoring. This time I was putting all the Core Data related code into a singleton and a couple of other small classes, making sure to use thread confinement since I did not want to get any of those dreaded EXC_BAD_ACCESS errors I had read so much about. How I did this is a discussion for another time though.

When the dust cleared I had a nice little set of classes that made it possible to manage my persistent data needs with just a few method calls. But this did prevent me from implementing all of the features. All in all it was a great experience and I learned a lot, even came away with some code I will make good use of in the future.

When we met on the final day to show our projects it was an impressive sight to see how my coworkers had turned their ideas into reality in such a short amount of time, especially while learning a new technology. Personally, I am completely hooked on Objective-C and will be spending a lot more time using it.

The iOS Hackathon is Complete!

If you read any previous posts on our iOS Hackathon, you’ll know that our engineering team spent one week learning Objective-C and the iOS SDK, and another week hacking out individual iOS projects. At the end of that second week, we all met together in a conference room, along with the judges, comprised of Kevin (our fearless CTO) and our Product Management team.

Each of us was given a maximum of five minutes to present our projects to the group. It was amazing to see all that we had accomplished in one short week in a new programming language and on a new platform.

After the presentations were over, the judges went in a separate room and discussed their findings. When they returned, they made the announcement. Two engineers received honorable mention: Brandon and Matthew. And the winner was Destin’s “Hau Much Is It?” app, which is similar to “Price Is Right” but uses products from live HauteLook events. His prize: an Apple Developer account! Now, Destin can release his own iOS apps to the App Store. (Note to Destin: remember, with great power comes great responsibility. Use this account to publish amazing apps.)

After the judging was complete, I was chatting with Mark Geller, who is the Product Manager responsible for our iOS and Android apps, and for our mobile site. In addition to being an amazing Product Manager, Mark also writes code on the side, and he’s quite knowledgeable in iOS. Regarding the crash course and hackathon, Mark said, “On a scale of 1 to 10, this was a 10. It probably couldn’t have gone any better!”

To view and download the iOS Hackathon apps, go here on an iOS device.

HauteLook’s iOS Crash Course

In a previous post I described our iOS Crash Course / Hackathon. Here, I’ll review the weeklong crash course.

On Monday 12/3, all of the crash course participants gathered together in one of our conference rooms. Jamie, our Sr. iOS Engineer, had graciously spent the weekend putting together a simple demo app that made an API call to our events service and displayed a list of current events. Jamie started to walk us through how the app was structured and how it worked, and most of us quickly realized that we were lost. Many in the group had never seen Objective-C, so the syntax was difficult to follow.

To get everybody familiar with the language, we watched a series of awesome videos on Lynda by Simon Allardice. These lasted a total of around 6.5 hours, so we watched some that afternoon and the rest on Tuesday morning. At this point, after watching those videos, we all had a decent understanding of Objective-C syntax and the basics of how it works.

Armed with this new knowledge, we went through the iOS Apprentice series of tutorials from Ray Wenderlich’s site. Mark Geller, our Head of Product for Mobile Marketing, recommended that we use this series. We found these tutorials to be very helpful and educational, and we too highly recommend them to anybody who wants to learn the basics of iOS development.

Now it’s Friday afternoon. Our crash course is over. All participants seem to believe that they’ll be able to take on their hackathon projects. We just met to cover these projects, and here’s what everybody is going to work on:

  • Brandon – Tap-hold to select item for custom catalog
  • Ethan – Facebook register / login
  • Dallas – Tap-hold on catalog page to show size / color picker and add to cart
  • Chris (me) – Show member’s order history and let member rate and review items
  • Matthew – Two-across grid for catalo with title and price
  • Destin – Hautelook “Price is Right” game
  • Bill – Magic 8 Ball product picker
  • Jimmy – Mobile stats
  • Jamie / Allen – Tool for HauteLook photo studio to pull images from old events

Next week we’ll start the Hackathon, so I’ll provide another update soon.

The Finish Line

One of the biggest problems struggling development teams have is shipping code. Most projects seem to have the same story of why they are late. Lots of planning and requirements gathering is done ahead of time to set the team up. The project starts off running smoothly and all the metrics are hinting that the project is on track. Confidence is high among the team and the managers are happy. Once you get to close to the 80% mark, then the progress starts to slow down. The last few tickets or a large bug end up taking 4-5x the estimated time. As a result, the deadline is missed. Missed by a mile.

This pattern where the last few issues of a project taking up a large portion of the work effort is common. There is always one bug that needs to be fixed or one feature that is a business owner wants to add. It is easy to find excuses to not ship code. This anomaly, where 20% of the project requires the most amount of effort, is known as the Pareto Principle. Development process like Agile and Lean have sprung up as a direct result of these challenges. Many companies are now familiar with the term MVP to release early and fight off scope creep.

I have found that process, no matter how much or how little, is unable to solve this problem of meeting deadlines. HauteLook uses two Agile methodologies: Scrum and Kanban. We have given multiple presentations on what MVP means. None of it has been as effective as increasing the amount communication among team members. When a team is communicating well they are implicitly working together. They are much more effective at problem solving and cutting through red tape. Communication also increases the sense of ownership over a project amongst the team members. When someone is included in a team discussion they feel like they belong. There is a sense of camaraderie. They are now much more likely to participate in future discussions and take initiative.

You can usually tell the difference between a team that communicates and a team that just goes through the motions by watching the daily stand-up. A team that is not communicating will usually be giving a status report to the scrum master. As each person gives their status update the other team members are zoning out because they are not sure what that person is really talking about. You will notice members of the team checking their phones or staring off into space. A team that communicates will be giving updates to the other team members, not the scrum master. A communicating team is engaged and they are making plans to fix any blockers that come up. There is a sense of urgency.

One of the best ways to facilitate an increase in team communication is to remove process. I first looked to our tools. We use JIRA at HauteLook to track the progress of sprints. It is very easy to add a lot of process to JIRA and think it will increase team efficiency. It is usually just the opposite. Too often process gives people a false sense of security that the process itself is going to solve the problems. Process does not solve problems; people do. When I am leading a team, a ticket is not ready for QA when a JIRA status is changed. A ticket is only ready for QA when the developer contacts the QA people on the team and informs them the ticket is ready. This communication can take place over email, but I encourage a more personal form of communication (in person, phone or instant message). The difference is that when a person changes status they are not expecting any feedback. They place the responsibility on the “process” working. When a person reaches out to another person they are expecting an acknowledgement of some sort. This sort of common interaction among team members creates a positive foundation and allows them to hold each other accountable in a constructive manner.

I double down on intra-team communication when a project is approaching the deadline. It is too easy for some people on the team to feel like they are done with the project when all the issues assigned to them are complete. When someone feels that sense of accomplishment they do not want to have to circle back to work on the project more. That news often comes as a surprise to them and now working on the project feels like an extra burden. By continuing to communicate with every member of the team on what the overall status of the project is they are much less likely to get that false sense of accomplishment. Instead, they are more likely to participate in dragging the project over the finish line and meeting the deadline as a team. Then, and only then, do they feel a real sense of accomplishment.

Making the Case Moving from Selenium RC to WebDriver

Over the past year we have been using Selenium RC to drive our front end testing automation. We were quite happy using RC as it’s well supported, documented and stable. Frankly it does what we need it to do. After learning about the advantages of Selenium 2.0 WebDriver, we decided to research the benefits and determine how much effort it would be to convert our existing automation written for Selenium RC over to the WebDriver API. This leads to the obvious question, why would you change? If it’s broke don’t fix it. The obvious answer is because that’s what we do in technology, try new technologies and strive for continuous improvement. We enjoy playing with new stuff. Let’s discuss the advantages and considerations we faced when moving to WebDriver.

Selenium RC vs WebDriver
Selenium is a testing framework used for automating testing across multiple browsers. First there was Selenium RC, developed by Jason Huggins that allows for writing automated web UI tests using a variety of languages including Java, PHP, C#, Perl Python and Ruby for Javascript enabled browsers. The API provides a feature rich set of functions that can be used to test a wide variety of web applications and browsers. It’s great for asserting and locating UI elements. Selenium RC contains a javascript based engine that injects javascript into the browser to drive testing. Javascript is used to control the browser instead of using the native browser controls.

Selenium 2.0 was introduced by Simon Stewart and allows for direct interaction with the browser’s native automation controls. The API is easier to understand than Selenium RC and a bit easier to set up in that you don’t need to start any additional processes like the proxy server used with Selenium RC. For those looking to explore WebDriver , Selenium 2.0 has both WebDriver and Selenium RC bundled together for language specific client drivers including Java, C#, Ruby and Python. Other non-supported webdriver bindings can be found for .NET, Perl and PHP. There is the added advantage of being able to to call a WebElement and perform several actions on it instead of calling that WebElement each time you want to perform an operation.

Our conclusion is since we can exploit a browser’s native controls using WebDriver, this should provide us with additional benefits by employing a more natural style of automation testing by rooting out the js interaction. Furthermore, as our automation was being utilized by more team members we needed a way to provide an easy setup for our QA group to run tests locally. With Selenium RC you need the proxy server. With WebDriver, you will need the browsers you want to test setup locally on your test machines although the proxy server was never really difficult to setup. Additionally we needed an opportunity to make our testing harness code more modular and easier to maintain.

Justification
Converting your automation testing harness can take a significant effort depending on how much you currently have in place. You may have hundreds of scripts that need to be changed. Your team has already invested significant time and dollars into developing and fine tuning your existing test. Can you justify this conversion to upper level management? This will cost time, money and will require people power drawing them away from other tasks. In our case, we didn’t have hundreds of scripts to update but our level of effort for converting scripts from RC to WebDriver was certainly bigger than a bread basket. So why take something that works relatively well and convert it to something new that works relatively well? I needed more justification before making that leap.

A discussion was held with the automation engineers where we determined that before we can make the conversion, we should refactor our automation code so that it’s more modular making it easier to maintain and execute for QA staff. One of the engineers mentioned using page objects. Page objects are useful for abstracting out what needs to be tested in your web UI. In other words you create a class which contains the objects/elements you want to test in a specific part of your UI. Take for instance our registration page. We created a class that contains all the objects we want to test for our registration page:
class RegistrationObject {
public $locators = array(
'firstName' => array('id', 'real_firstname'),
'lastName' => array('id', 'real_lastname'),
'registerButton' => array('id', 'register_btn')
);

public function enterRegistrationData($first_name = null, $last_name = null) {
$this->firstName->sendKeys($first_name);
$this->lastName->sendKeys($last_name);
}

public function register() {
$this->registerButton->click();
$homePage = Application::create('homePage');
return ($homePage->isDisplayed());
}
}

In your test class you can call the page objects you want to test:
class RegistrationVerification extends WebDriverTestCase {
public function testRegistrationVerification() {
$firstName = “qa”;
$lastName = “automation”;
$registrationPage = Application::navigateTo(‘RegistrationPageObject’);
$registrationPage->enterRegistrationData($firstname, $lastName );
$this->assertTrue($registrationPage->register(), “Home Page is not displayed after registration”);
}
}

Note: Our WebDriverTestCase extends PHPUnit_Extensions_Selenium2TestCase

This has several added benefits:
1. Easy to reuse code – just call the page object that contains what you need to test
2. When your web UI changes just update the page object. Ideally your tests should never need to change unless there is new functionality that needs to be tested.

Start Small
Experience has taught me to start small when making changes to automation framework. There is nothing like discovering midway through that this is not what you wanted, that it falls short of your expectations, or you have taken on more than you can handle. Before making significant changes to your automation, start with preparing a plan by having a discussion with the automation team. Identify weaknesses within your current system and strategize what you want to be the big picture. In our case we wanted to take advantage of Selenium’s 2.0 WebDriver features, make the automation more modular thus easier to maintain and easier for QA testers to use.
At this point we have established our justification, a plan and a strategy and are ready to begin work. In keeping with our tech departments MVP philosophy on software development we utilized a build, measure and learn approach taken from the Lean Startup Methodology. First we identified a single automation script written in Selenium RC to convert to WebDriver. This involved:
1. Investigating what WebDriver library we were going to use
2. rewriting the script to use the WebDriver API from step 1
3. Create a page object
4. Compare the performance between the Selenium and WebDriver scripts

Once the first script was complete and executed we held a follow up meeting to discuss the results. We had uncovered several issues: The PHP Web Driver library we chose was too limited. Creating page objects proved some difficulty for those not familiar. The performance was significantly worse than Selenium RC.
We repeated the build, measure and learn process by investigating and addressing each one of these areas by retrofitting the existing script with new WebDriver libraries phpunit_selenium by Sebastian Bergmann, reducing POST/GET requests then we reconvened to discuss the results which were much more promising than our first iteration. We found a WebDriver library that served our needs, performance was dramatically improved by reducing POST/GET requests on and the automation engineers became much more comfortable with creating Page Objects. At this point we felt we had a solid base where we could continue converting existing RC scripts to WebDriver.

Summary
There are many questions to consider when deciding to change your automation framework and should not be taken lightly. Have a strategy and plan before making the case to upper management. I would recommend that for anything. In our case it was to make the system easier to maintain, achieved by creating a more modular design and natural testing. Start small and work in iterations by defining what is to be built and measured in the iteration. At the end of the iteration, hold a retrospective to share what was learned and consider if this effort is worth proceeding or perhaps a pivot needs to occur. With adequate discussion, strategy and proper planning with this endeavor you and your team can be well positioned to succeed.


References:

http://stackoverflow.com/questions/8560462/what-are-the-advantages-of-selenium-webdriver-over-selenium-rc

http://seleniumhq.org/docs/appendix_migrating_from_rc_to_webdriver.html

The Lean Startup – by Eric Ries

https://groups.google.com/forum/?fromgroups#!topic/selenium-users/SnnteaKGNHs

http://seleniumhq.org/docs/01_introducing_selenium.html

http://code.google.com/p/selenium/wiki/RemoteWebDriver

http://code.google.com/p/php-webdriver-bindings/

https://github.com/chibimagic/WebDriver-PHP/

http://blog.josephwilk.net/cucumber/page-object-pattern.html

Participating in 24PullRequests

This time of year is all about giving and open source is no exception. I recently saw some tweets about a project called http://24pullrequests.com/ and was immediately interested. It is an effort to get more people involved in the open source community and the community for the software you commonly use. From the website:

24 Pull Requests is a little project to promote open source collaboration during December.

The idea is basically “Send a pull request every day in December leading up to Christmas”, encouraging developers to give back to open source with little gifts of code throughout December.

This is a great way to discover new projects out there. Once you login, using your github account, you will be given project suggestions. You can also view all the projects and filter by language. If you are interesting in learning a new language, one great way is to fix bugs and update documentation.

I have been doing a lot of development with Guzzle lately. Trying to abstract away the HTTP requests can get pretty hairy when API’s do unusual things. I have been contributing back to the Guzzle documentation to make it easier for the next person to grok just how awesome Guzzle is. Github makes it so easy to update documentation that you usually do not need to even make a local clone of the repo. You can edit it right through the github website.

I have also suggested a number of projects for people to contribute to, including:

If you want to suggest an open source project yourself, go to http://24pullrequests.com/projects/new and fill out the form.

Happy hacking!

HauteLook Returns Processing

Why it was rebuilt, how we did it and what we learned in the process

Processing returns for an online retailer is one of the most complicated things you can attempt to figure out next to actually selling the products. You have to determine what items you will allow to be returned, when they can be returned, how much if anything they will be charged for the service, if they need to ship the item back and where to, how to pay for the return shipment fees, who will pay for these fees and when the customer will actually get their money back. Then, after all that is done you still have to account for it all for tax purposes.

Naturally when HauteLook implemented a returns processing system they did not get it all correct the first time, and as time passed and more requirements were added the system grew to point where it was obvious to many that a major rewrite was in order.

As we could see it, there were two major problems that needed to be addressed:

Accountability: The original system allowed our agents to manually change certain values, such as how much of a customer’s original freight was returned. This was done because sometimes a customer had a good reason for the return, sometimes it wasn’t their fault and they shouldn’t have to pay for the shipping. While this was a necessary feature, the problem was with how it was implemented. Returns were created and then at some later date they were modified. We didn’t know who did the modification or when. Also if the modification was done after the end of a month, our Finance Department would have incorrect totals for how much was returned in that month which led to issues with reporting taxes.

Flexibility: The old system was actually built in two places, on our e-commerce site and also in our administrative site and was built to be used both directly and via an API. Originally the code in these locations was very similar if not identical, however as time progressed the code mutated in different directions on both sites. Business rules would be added to one or another site as they were needed. Bug fixes would not always be fixed in both places. And the code in general went from relatively easy to read chunks to massive functions spanning hundreds of lines that were nearly impossible to decipher except by their original creators. Making even minor changes was extremely difficult to do without breaking something else.

Along with these major problems were a slew of bugs ranging from simple to very elusive that had yet to be addressed in the old system. The challenge we had was how to simultaneously build a new system while fixing the old system and keeping everyone happy during the process.

New ideas

In order to address the issue of accountability we created the concept of a ‘Financial Adjustment’. A financial adjustment was a new kind of line item on a return that did not have to be associated with a particular order item. It was simply a representation of money being returned to the customer. Now, when we completed a return, which previously consisted only of return items that were associated with a particular order item, we could calculate the values that this item should have and leave them alone, never to be changed. Any modifications to the return would be tracked as financial adjustments. These financial adjustments could live on a return modifying the overall values of the return or they could live on their own and be created at any time in the future without having to modify any historical values. Also these adjustments would track who made them and when. It allowed us to do something like this:

March: A customer returns an item for $10; $10 is deducted from our revenue for the month of March.
April: The customer is refunded an additional $5 to cover shipping costs because it turns out the item was damaged and they should not be responsible for these costs. An additional $5 is deducted from the revenue for April (instead of taking it out of the return created in March).

And thus we have accountability!

The flexibility issue would be addressed during the rewrite of the code. In this rewrite we decided to use the following techniques:

  1. The new system would be built in one location and accessible via a common REST API. All requests coming from our administrative, e-commerce and mobile platforms would use this service in the same way.
  2. We would use the Dependency Injection pattern so that the database could be decoupled from the actual business logic and also to reduce dependencies throughout the new code.
  3. The Chain of Command pattern would allow us to isolate the actions that needed to be done and allow us to easily change them without breaking other modules if necessary.
  4. And finally, Unit testing would be used to a much greater extent than the old system which would allow us to make changes and identify problems quicker.

The Rollout Process

Rather than building the entire system and then turning it on and hoping for the best we decided to release this project to the world in small steps. We did this mainly to minimize the possibility of breaking the entire site and also to ease our Member Care people into the new system. We started first with what we called cancellations. Cancellations were orders that were returned before they shipped and were therefor a bit easier to process. Then we moved on to Inventory returns (ones that we expected to be shipped back into our inventory) and finally on to Financial adjustments.

Each of these major revisions was also followed by several smaller releases that fixed new bugs as they were introduced. After everything was said and done, the development of the new system took about 10 months to complete.

The Replay System

Another important step we decided to take, after Cancellations had already been released, was to implement a returns replay system. What the replay system did was take returns that had already been processed with the old system and rebuild their data in a temporary database so that they could be re-processed with the new system. We then used PHPUnit to compare the results of the live returns vs. the test ones to determine if our new system was functioning properly.

While this system did help us iron out many of the initial bugs before we went live, it was not perfect. This is because it started with the assumption that the old system processed all the returns properly which was of course not the case. Because of this we started adding more and more exceptions to this system until it got to the point that it really wasn’t very useful anymore.

Structural Limitations

As we developed the new system we began to realize a number of problems that were outside of our control. One of these problems was the Order system and rounding.

In our orders system we stored order items in rows in the database according to their specific product SKU, and the quantity of those items ordered, however in the returns system we did not track the quantity, opting to have an individual row for each item returned. This led to the inevitable rounding issue, much of the calculation in the new returns system deals with how to properly determine where a fraction of a cent might be applied. While this may seem trivial, the difference of one cent can be the difference of your transaction being accepted or not and that is a big deal.

We also introduced some limitations of our own, for example, when we added Financial Adjustments we decided to make them completely independent of any physical items being returned. This was great from a conceptual point of view as everything was only related to the actual order/transaction that was made, however this prevented us from being able to determine exactly what had been done on an item by item basis. We could for example determine that $5 of freight was returned on order #1234 but if that order had 10 items we couldn’t tell which of those items’ freight was actually returned.

What We Learned

For me the main things I took away from this project were how to correctly do unit testing and the importance of clean code. The clean code part unfortunately was something I learned in the latter half of the project after reading Clean Code, A Handbook of Agile Software Craftsmanship by Robert C. Martin. This book opened my eyes to some of the bad habits that I had learned and incorporated into the project up to that point.

I also learned that test driven development is easier said than done. At the beginning of the project I had abandoned TDD fairly quickly because of how much longer it seemed to be taking me to get things done. I later realized that there were two reasons for this. First, I did not have a good and proper understanding of how unit tests should be done and second, my code was not in clean, easy to digest blocks. I would advise anyone attempting TDD to first read Clean Code and then write as many unit tests as you can for existing code before you start.

Code Freeze iOS Hackathon

Every year around the holiday season, we enter “Code Freeze”, a time where we don’t release any code changes to our e-commerce site or API unless they are of utmost priority and are low-risk. Code Freeze is a time where we can all relax a little, breathe easier, and have some fun. It’s also a great time to pay off tech debt, but that’s the topic of another post that’ll probably never be written.

This year, one of the cool things we’re doing during Code Freeze is an iOS hackathon. We’re basically shutting down development for two weeks. Why? To have fun. We’re proud of our hacker culture, and giving the team a chance to learn and try out a new technology is something that we’re really looking forward to. And who knows? Maybe we’ll get lucky and find that one or more of our engineers can move over to the iOS team. It’s frustratingly difficult to find and hire quality iOS engineers, so this could be a great opportunity to fill a much-needed position on the iOS team.

iOS Crash Course

The concept is simple. Any of our engineers who are interested (and they all are) will spend a week taking a crash course on iOS, presented by Jimmy, our Sr. Manager of iOS Development, and Jamie, our Sr. iOS Engineer. This crash course will consist of classroom time, independent study, and hands-on experience. We’ll likely utilize educational tools such as Stanford on iTunes U and/or the tutorials from Ray Wenderlich.

The Hackathon

After the week-long crash course, we’ll spend a week hacking out some simple but amazing iOS apps. We haven’t yet decided on exactly what these apps will be, and of course, we’ll get feedback from all of the participants regarding what we should work on. One idea is that we can make several tools which port functionality from our Admin site to iOS. This is awesome for a couple of reasons. First, we should be able to hook up to existing HauteLook APIs. Second, we’ll end up with a suite of tools that our co-workers throughout HauteLook can actually use to do their jobs on a daily basis. Courtesy of the HauteLook Engineering team!

We’ll Keep You Posted

Within a couple of weeks, this blog should have posts by multiple engineers, detailing their experiences with the iOS crash course and hackathon. Hopefully everybody will have fun. And maybe we will find our next iOS engineer. I’m keeping my fingers crossed!

Page 1 of 41234»