MessageBoard, a PHP & MySQL web application with a Doctrine ORM back-end, was launched last Friday, right on deadline. This concludes 7 month-long phases of development.

I think the ingredients essential to delivering this application on-time and within budget were:

  • Partnering with a user experience expert who designed the mock ups and wrote the bulk of HTML, CSS, and Javascript
  • Doctrine ORM: previously we hand-crafted SQL which was error-prone and occasionally introduced security vulnerabilities.
  • Month-long development iterations with a tested, working product delivered at the end of every month.

Looking forward, there is an increasing need at our campus for an Academic Personnel Review web application, which will transform a paper process into a fully online process. The major motivation behind this project is to reduce the massive amount of staff time that goes into coordinating this process in each campus unit.

This application will very likely be built in Java!

, , ,

Just wanted to post an update to my previous blog entry about Doctrine ORM gotchas. Since 1.0.4 was released, a seriously limiting bug was fixed in the SoftDelete template. This bug was preventing typical performance optimizations that used LEFT JOINs to reduce the number of database queries. The idea is that a page generally loads much faster by executing few efficient JOIN’ed queries than many single-table queries (do your joins in MySQL, not PHP!)

I had posted a workaround to this bug ($query->addWhere(”deleted = 0 OR deleted IS NULL”) to all of your LEFT JOINs). This was cumbersome and I felt violated the principle of the SoftDelete event listener.

With this bug resolved, I’ve been more freely adding custom finders for specific pages. One page in particular (the MessageBoard thread index) went from 12 seconds to 4 seconds for a very large data set. The number of DB queries also was cut from 1200+ to about 300.

Now I just have to get the page down to 10 queries and we can call it optimized.

, , ,

Doctrine ORM is a PHP library that implements the ActiveRecord pattern we have all grown to love.  I’ve been using it for the past 7 months and feel it is one of the reasons we were able to deliver the MessageBoards web application on-time and within budget.

This doesn’t mean using Doctrine has been all flowers and sunshine: Doctrine will kick you in the face when you’re not paying attention.

Today I’d like to revisit all of those black eyes and bloody noses in the hope of helping fellow developers avoid the same missteps I made.  Doctrine is complex and quirky, and has some unanticipated architectural “features” that are not well documented.

Use Record::toArray() and Record::fromArray() to store/retrieve Doctrine objects from the session.

  • Save space in the session store by adding only the column attributes of Record objects to the session.
  • The session will quickly fill up otherwise, as Doctrine adds considerable bulk to model objects.

Improve performance by extending Doctrine_Table and implement custom DQL queries for complex and frequently used queries.

  • If the controller or view will need a record’s related record, use a DQL query to join with the related table.

Optimize performance by getting to know and love Doctrine_Connection_Profiler.

  • Add the connection listener at the beginning of execution and print SQL queries at the end of execution in order to identify areas of effective performance optimization.
  • Example code that adds the listener and renders query events as HTML:
    
    // Set the connection listener
    
    $profiler = new Doctrine_Connection_Profiler();
    Doctrine_Manager::connection()->setListener($profiler);
    
    // Code goes here...
    
    // Render database connection events as HTML:
    
    $query_count = 0;
    $time = 0;
    echo "<table width='100%' border='1'>";
    foreach ( $profiler as $event ) {
        if ($event->getName() != 'execute') {
            continue;
        }
        $query_count++;
        echo "<tr>";
        $time += $event->getElapsedSecs() ;
        echo "<td>" . $event->getName() . "</td><td>" . sprintf ( "%f" , $event->getElapsedSecs() ) . "</td>";
        echo "<td>" . $event->getQuery() . "</td>" ;
        $params = $event->getParams() ;
        if ( ! empty ( $params ) ) {
              echo "<td>";
              echo join(', ', $params);
              echo "</td>";
        }
        echo "</tr>";
    }
    echo "</table>";
    echo "Total time: " . $time . ", query count: $query_count <br>\n ";
    

Effectively mitigate performance issues with memcache.

  • The query and result cache can drastically offset Doctrine’s performance overhead.
  • If you already have memcached running, this is one of the most cost-effective performance tweaks you can do.
  • Note: I received mysterious fatal errors when using INDEXBY in DQL queries. After removing the INDEXBY, the errors stopped.

Play nice with fellow coders or testers by automating database migrations.

  • Add a git merge hook that runs the Doctrine migration.
  • Alternatively, check if a migration is needed on every page view while in development mode.

Play nice with other web applications by prefixing database tables.

  • Set the table name prefix by calling $this->setTableName(’zzz_model_name’), where ‘zzz’ is the tool’s prefix.

Create a “resource-oriented” URL structure that closely follows the application’s models.

  • This is borrowed from the RESTful architecture, and not necessarily Doctrine-specific.
  • For example an HTTP GET on http://site.com/messageboard/m34/f11/ would display forum ID 11 in messageboard 34 in tool “messagebord”.

Use a workaround when using LEFT JOINs on models with actAs(’SoftDelete’) behavior.

  • SoftDelete will automatically add the WHERE condition “deleted = 0″ to all queries.
  • This prevents queries with LEFT JOIN from returning a row where “deleted IS NULL”.
  • Either use INNER JOIN instead, or add the following to DQL queries: $query->andWhere(’deleted = 0 or deleted IS NULL’);

Timestampable cannot be disabled temporarily, causing challenges when importing data with dates.

  • Doctrine provides no way to easily disable or override the timestamp behavior in order to import a pre-existing date.
  • Until this behavior is resolved, try using this patch to set the ‘disabled’ option of Timestampable.

Use “cascade => array(’delete”)” to propagate soft deletes through model relations.

  • The faster onDelete => ‘CASCADE’ performs the delete in MySQL, which does not set the deleted flag.

Put authorization code in one place, when possible.

Implement checkbox plus text input as two columns in a model.

  • For example:
    Require password:
  • This approach simplifies validation of optional attributes.

Use $model->setAttribute(Doctrine::ATTR_COLL_KEY, ‘id’) to key collections off of the primary key.

  • If this attribute is not set, Collections will be indexed starting from 0 and counting upwards.
  • This can simplify controller logic.

Use Doctrine_Pager only for the most basic views.

Use actAs(’NestedSet’) to model hierarchies that are read more frequently than written.

Use unix timestamps in Timestampable columns to ease formatting.

  • Using a datetime works fine if the view never needs to change how a datetime is displayed.
  • A unix timestamp allows for flexibly changing how dates are rendered, e.g.: “Jan 1st 2008″ or “Yesterday”.
  • Example actAs() code:
    $this->actAs('Timestampable', array(
        'created' => array('name' => 'created_at',
            'type'    =>  'integer',
            'format'  =>  'U',
            'disabled' => false,
            'options' =>  array()),
        'updated' => array('name'    =>  'updated_at',
            'type'    =>  'integer',
            'format'  =>  'U',
            'disabled' => false,
            'options' =>  array())));
    

For multi-step forms, add a ’state’ column to aid in validating each step.

  • In the model’s validate() function, use the state column to switch between validation logic.
  • For instance, in state 1, validate columns a and b. In state 2, validate columns a, b, c, and d. In state 3, validate the complete object.

I hope this list saves some heartache on what is really a very elegant ActiveRecord implementation in PHP!

, , ,

Creating a minimal LAMP stack in OpenVZ:

Start with a pre-created centos-5 OpenVZ template & install required packages:


# Create centos-5 OpenVZ container:
vzctl create 1056 --ostemplate centos-5-x86_64-minimal
vzctl set 1056 --ipadd 192.168.0.56 --nameserver 123.123.123.123 --save
vzctl start 1056

# Update software & install packages:
vzctl 1056 install yum
vzctl enter 1056
yum upgrade

# Install packages -- the ".x86_64" tells yum to only
# install the 64-bit packages and not the i386 packages.
yum install vim-enhanced.x86_64 mysql.x86_64 mysql-server.x86_64 \
httpd.x86_64 httpd-devel.x86_64 wget.x86_64 which.x86_64 \
php.x86_64 make.x86_64 gcc.x86_64 gcc-c++.x86_64

# Clean up leftover files from yum:
yum clean all

Next, tune Apache to fit our development 256MB OpenVZ container. If you have more memory dedicated to your container, consider increasing these settings. Edit the prefork MPM section of /etc/httpd/conf/httpd.conf:

<span class="__mozilla-findbar-search" style="padding: 0pt; background-color: white; color: black; display: inline; font-size: inherit;">&lt;</span>IfModule prefork.c<span class="__mozilla-findbar-search" style="padding: 0pt; background-color: white; color: black; display: inline; font-size: inherit;">&gt;</span>
StartServers       2
MinSpareServers    1
MaxSpareServers   8
ServerLimit      8
MaxClients       8
MaxRequestsPerChild  4000
<span class="__mozilla-findbar-search" style="padding: 0pt; background-color: white; color: black; display: inline; font-size: inherit;"><span class="__mozilla-findbar-search" style="padding: 0pt; background-color: white; color: black; display: inline; font-size: inherit;"></span></span>&lt;/IfModul<span class="__mozilla-findbar-search" style="padding: 0pt; background-color: white; color: black; display: inline; font-size: inherit;"><span class="__mozilla-findbar-search" style="padding: 0pt; background-color: white; color: black; display: inline; font-size: inherit;">e&gt;</span></span>

Let’s lock down the MySQL root user before starting up services:

mysql -u root mysql
mysql> update user set password=password('mynewsecurepassword') \
where user='root';
mysql> flush privileges;
mysql> exit

Start services and ensure that services start when the machine boots up:

chkconfig --levels 345 httpd
chkconfig --levels 345 mysqld
service httpd start
service mysqld start

Finally: test!

This will give you a slimmed down LAMP stack, suitable for running small web applications on top.

, , , ,