PHP-CRUD-API now supports authorization and validation

Another milestone is reached for the PHP-CRUD-API project. A project that aims to provide a high performance, consistent data API over REST that is easy to deploy (it is a single PHP file!) and requires minimal configuration. By popular demand we have added four important new features:

  1. Tables and the actions on them can be restricted with custom rules.
  2. Access to specific columns can be restricted using your own algorithm.
  3. You can specify “sanitizers” to, for example, strip HTML tags from input.
  4. You can specify “validators” functions to show errors on invalid input.

These features are built by allowing you to define callback functions in your configuration. These functions can then contain your application specific logic. How these function work and how you can load them is explained below.

Table authorizer

The following function can be used to authorize access to specific tables:

/**
 * @param action    'create','read','update','delete','list'
 * @param database  name of your database (e.g. 'northwind')
 * @param table     name of the table (e.g. 'customers')
 * @returns bool    indicates that access is granted  
 **/
  
$f1=function($action,$database,$table){
  return true; 
};

Column authorizer

The following function can be used to authorize access to specific columns:

/**
 * @param action    'create','read','update','delete','list'
 * @param database  name of your database (e.g. 'northwind')
 * @param table     name of the table (e.g. 'customers')
 * @param column    name of the column (e.g. 'password')
 * @returns bool    indicates that access is granted  
 **/
  
$f2=function($action,$database,$table,$column){
  return true; 
};

Input sanitizer

The following function can be used to sanitize input for specific columns:

/**
 * @param action    'create','read','update','delete','list'
 * @param database  name of your database (e.g. 'northwind')
 * @param table     name of the table (e.g. 'customers')
 * @param column    name of the column (e.g. 'username')
 * @param type      type of the column (depends on engine)
 * @param value     input from the user (e.g. 'johndoe88')
 * @returns string  sanitized value
 **/
  
$f3=function($action,$database,$table,$column,$type,$value){
  return $value; 
};

Input validator

The following function can be used to validate input for specific columns:

/**
 * @param action    'create','read','update','delete','list'
 * @param database  name of your database (e.g. 'northwind')
 * @param table     name of the table (e.g. 'customers')
 * @param column    name of the column (e.g. 'username')
 * @param type      type of the column (depends on engine)
 * @param value     input from the user (e.g. 'johndoe88')
 * @param context   all input fields in this action
 * @returns string  validation error (if any) or null
 **/
  
$f4=function($action,$database,$table,$column,$type,$value,$context){
  return null;
};

Configuration

This is an example configuration that requires the above snippets to be defined.

$api = new MySQL_CRUD_API(array(
  'hostname'=>'localhost',
  'username'=>'xxx',
  'password'=>'xxx',
  'database'=>'xxx',
  'charset'=>'utf8',
  'table_authorizer'=>$f1,
  'column_authorizer'=>$f2,
  'input_sanitizer'=>$f3,
  'input_validator'=>$f4
));
$api->executeCommand();

You can find the project on Github.

Data-driven API design

We live in a world of APIs. LeaseWeb is an infrastructure provider that has taken on an API-first approach. This means that we first build an API for our products. Only after the API is finished we build the UI for the product and we do this on top of our own API. This “eat your own dog food” principle ensures a high quality API in which bugs are found before the customer encounters them.

Our “bare metal” (formerly: dedicated server) team has an API for switch port management. Opening a switch port may (dependent on the switch brand) take up several seconds. This is because (for instance) a SSH connection has to be made to the switch, then an XML command and answer has to be sent back and forth. Also, some switches may wait a few seconds to respond with a status, as they are trying to identify the link status, after the port is opened.

But what if a customer requests a “faster” API? We can’t make opening or closing a switch port go faster. What to do? If any API call would only modify a single record in the database, then the execution time would be predictable (and always low). To achieve this there are two API designs possible: “asynchronous” and “data-driven”. In this post we will explore both and show you why a data-driven API is a better choice for consistent APIs.

Synchronous API design

This is the naive implementation. It allows you to Open/Enable the switch port attached to the server with a synchronous call:

POST /bareMetals/{bareMetalId}/switchPort/open

Parameters:

bareMetalId (integer) - The id of the bare metal server

Example request:

POST /v1/bareMetals/1234/switchPort/open HTTP/1.1
Host: api.leaseweb.com
Accept: application/json
X-Lsw-Auth: 4e7d4f2d-e683-4192-a113-61dad6ac9a15

Example response:

HTTP/1.1 200 OK
Content-Type: text/html

{
	"switchPort": {
		"status": "opened",
		"switchNode": 1,
		"serverId": 1234,
		"serverName": "My Server"
	}
}

Note that this call may take several seconds to complete and does not follow the RESTful resource naming scheme.

Asynchronous API design

If the call is asynchronous then the request is the same:

POST /bareMetals/{bareMetalId}/switchPort/open

Parameters:

bareMetalId (integer) - The id of the bare metal server

Example request:

POST /v1/bareMetals/1234/switchPort/open HTTP/1.1
Host: api.leaseweb.com
Accept: application/json
X-Lsw-Auth: 4e7d4f2d-e683-4192-a113-61dad6ac9a15

Example response:

HTTP/1.1 200 OK
Content-Type: text/html

{
	"jobId": "f5825d99-62ed-433b-930b-1b9561fe4a6e",
	"createdAt": "2015-01-14T15:09:10+01:00"
}

The response is a job identifier that then has to be polled for status, like this:

GET /bareMetals/{bareMetalId}/jobs/{jobId}

Parameters:

bareMetalId (integer) - The id of the bare metal server
jobId (string) - The id of a job

Example request:

GET /v1/bareMetals/1234/jobs/f5825d99-62ed-433b-930b-1b9561fe4a6e HTTP/1.1
Host: api.leaseweb.com
Accept: application/json
X-Lsw-Auth: 4e7d4f2d-e683-4192-a113-61dad6ac9a15

Example response:

HTTP/1.1 200 OK
Content-Type: text/html

{
	"id": "f5825d99-62ed-433b-930b-1b9561fe4a6e",
	"status": 3,
	"statusDescription": "Failed",
	"createdAt": "2015-01-14T15:09:10+01:00",
	"updatedAt": "2015-01-14T15:09:12+01:00"
}

All reponses can be sent really fast as they are simple database lookups. It is important to realize that you need some sort of job manager in the back that does actually execute the jobs and updates the job status accordingly. Note that the commands do not follow the RESTful resource naming scheme.

Data-driven API design

When following data-driven API design you are simply modeling the database tables to facilitate the functionality. It allows you to have fast replies and also follows the RESTful resource naming scheme. You may get the status of the switch port attached to the server with a call like this:

GET /bareMetals/{bareMetalId}/switchPorts

Parameters:

bareMetalId (integer) - The id of the bare metal server

Example request:

GET /v1/bareMetals/1234/switchPorts HTTP/1.1
Host: api.leaseweb.com
Accept: application/json
X-Lsw-Auth: 4e7d4f2d-e683-4192-a113-61dad6ac9a15

Example response:

HTTP/1.1 200 OK
Content-Type: text/html

{
	"switchPorts": [
		{
			"status": "opened",
			"switchNode": 1,
			"serverId": 1234,
			"serverName": "My Server"
		}
	]
}

When you are opening the switch port you are actually inserting a record in the switchPortOpenRequests table with the standard POST as prescribed by the RESTful resource naming scheme, like this:

POST /bareMetals/{bareMetalId}/switchPortOpenRequests

Parameters:

bareMetalId (integer) - The id of the bare metal server

Example request:

POST /v1/bareMetals/1234/switchPortOpenRequests HTTP/1.1
Host: api.leaseweb.com
Accept: application/json
X-Lsw-Auth: 4e7d4f2d-e683-4192-a113-61dad6ac9a15

Example response:

HTTP/1.1 200 OK
Content-Type: text/html

{
	"id": "f5825d99-62ed-433b-930b-1b9561fe4a6e",
	"status": 1,
	"statusDescription": "Pending",
	"createdAt": "2015-01-14T15:09:10+01:00",
	"updatedAt": "2015-01-14T15:09:10+01:00"
}

Just like with the asynchronous model you can poll for status. Note that there is no job identifier, just a normal identifier, as the request is a normal resource that follows the RESTful resource naming scheme. You do still need to have a background process that watches the switchPortOpenRequests table to execute these requests when they are in “pending” state. This background process is responsible for updating both the switchPortOpenRequests and the switchPorts table with the correct status information.

GET /bareMetals/{bareMetalId}/switchPortOpenRequests/{id}

Parameters:

bareMetalId (integer) - The id of the bare metal server
id (string) - The id of the request

Example request:

GET /v1/bareMetals/1234/switchPortOpenRequests/f5825d99-62ed-433b-930b-1b9561fe4a6e HTTP/1.1
Host: api.leaseweb.com
Accept: application/json
X-Lsw-Auth: 4e7d4f2d-e683-4192-a113-61dad6ac9a15

Example response:

HTTP/1.1 200 OK
Content-Type: text/html

{
	"id": "f5825d99-62ed-433b-930b-1b9561fe4a6e",
	"status": 3,
	"statusDescription": "Failed",
	"createdAt": "2015-01-14T15:09:10+01:00",
	"updatedAt": "2015-01-14T15:09:12+01:00"
}

As you can see this last way of modeling does not only reduce every API call to be a database lookup, it also allows you to follow the (predictable) RESTful resource naming scheme.

Generating a data-driven API

The advantage of following an API naming scheme is that the API becomes predictable and can be generated. The PHP-CRUD-API project facilitates creating a full-featured API by uploading a single PHP file. This API will follow the resource naming scheme for your MySQL, PostgreSQL or MS SQL Server tables. With a data-driven API design this should not limit you to create any API functionality. Try it!

Symfony2 consistent routing

Software consistency in large teams is very important, because nobody likes whitespace or formatting commits or arguments about on which line brackets should be placed. Fortunately Symfony2 has a nice Standards Document that you can follow. PHP CodeSniffer can be loaded with the Symfony2 coding standard so that everybody can see the violations on the Continuous Integration (CI) server.

Unfortunately Symfony2 is not that strict everywhere. This lack of strictness allows a group of programmers to argue over and mess up a project. One of the discussions could be about naming routes. Symfony1 had a really nice option for “default routing”, allowing you to specify a single rule called “default” that had url: “/:module/:action/*”, but unfortunately that feature is not present in Symfony2. In Symfony2 we can either define routes individually or we can generate them using the CRUD generator (see my previous post). This is not a bad option as I will show below.

Naive routing

Routing to specific actions in controllers in Symfony2 can be put in a “routing.yml” file as described in the Routing section of the Symfony2 manual.

# app/config/routing.yml
product:
    pattern:   /product/
    defaults:  { _controller: AcmeDemoBundle:Product:index }

product_edit:
    pattern:  /product/edit/{id}
    defaults: { _controller: AcmeDemoBundle:Product:edit }

It is easy to understand that this way the file becomes large and unless people agree on a very strict naming policy it needs to be consulted every time you want to make a link from a view or controller to any action.

CRUD generated routing

After using the CRUD generator to generate scaffolding for an Entity called “Product”,

maurits@pc:~/project$ app/console generate:doctrine:entity --entity=AcmeDemoBundle:Product
...
Configuration format (yml, xml, php, or annotation) [annotation]:
...
maurits@pc:~/project$ app/console generate:doctrine:crud --entity=AcmeDemoBundle:Product
...
Do you want to generate the "write" actions [no]? yes
...
Configuration format (yml, xml, php, or annotation) [annotation]:
...
Routes prefix [/product]:
...

I edited my routing to include this rule:

# app/config/routing.yml
demo_bundle:
    resource: "@AcmeDemoBundle/Controller/"
    type: annotation
    prefix: /

This is the only routing entry you need per bundle when using (CRUD generated) routing annotation in your controllers. The “@Route” and “@Template” annotations can be seen below:

/**
 * Product Controller
 *
 * @Route("/product")
 */
class ProductController extends Controller
{
    /**
     * Lists all Product entities.
     *
     * @Route("/", name="product")
     * @Template()
     */
    public function indexAction()
    {
        $em = $this->getDoctrine()->getManager();
        $entities = $em->getRepository('AcmeDemoBundle:Product')->findAll();
        return array('entities' => $entities);
    }

    // ...
}

Your routes will look like:

maurits@pc:~/project$ app/console router:debug
[router] Current routes
Name                                                   Method Pattern
...                                                    ...    ...
product                                                ANY    /product/
product_show                                           ANY    /product/{id}/show
product_new                                            ANY    /product/new
product_create                                         POST   /product/create
product_edit                                           ANY    /product/{id}/edit
product_update                                         POST   /product/{id}/update
product_delete                                         POST   /product/{id}/delete

Default Routing in Symfony2

But would it not be prettier when the only routing entry you needed (per bundle) was this one?

# app/config/routing.yml
demo_bundle:
    resource: "@AcmeDemoBundle"
    type: default
    prefix: /

And your code would NOT contain “@Route” and “@Template” annotations and look like:

/**
 * Product Controller
 */
class ProductController extends Controller
{
    /**
     * Lists all Product entities.
     */
    public function indexAction()
    {
        $em = $this->getDoctrine()->getManager();
        $entities = $em->getRepository('AcmeDemoBundle:Product')->findAll();
        return array('entities' => $entities);
    }

    // ...
}

Your routes could look like this:

maurits@pc:~/project$ app/console router:debug
[router] Current routes
Name                                                   Method Pattern
...                                                    ...    ...
acme_demo.product.index                                ANY    /product/index.{_format}
acme_demo.product.show                                 ANY    /product/show/{id}.{_format}
acme_demo.product.new                                  ANY    /product/new.{_format}
acme_demo.product.edit                                 ANY    /product/edit/{id}.{_format}
acme_demo.product.delete                               ANY    /product/delete/{id}.{_format}

To achieve this we needed to combine the “new” & “create” and “edit” & “update” actions and also remove
the “$request” argument from the “delete” action and replace it by a simple “$request = $this->getRequest();” call. So this code:

    /**
     * Displays a form to edit an existing Product entity.
     *
     * @Route("/{id}/edit", name="product_edit")
     * @Template()
     */
    public function editAction($id)
    {
        $em = $this->getDoctrine()->getManager();

        $entity = $em->getRepository('AcmeDemoBundle:Product')->find($id);

        if (!$entity) {
            throw $this->createNotFoundException('Unable to find Product entity.');
        }

        $editForm = $this->createForm(new ProductType(), $entity);
        $deleteForm = $this->createDeleteForm($id);

        return array(
            'entity'      => $entity,
            'edit_form'   => $editForm->createView(),
            'delete_form' => $deleteForm->createView(),
        );
    }

    /**
     * Edits an existing Product entity.
     *
     * @Route("/{id}/update", name="product_update")
     * @Method("POST")
     * @Template("AcmeDemoBundle:Product:edit.html.twig")
     */
    public function updateAction(Request $request, $id)
    {
        $em = $this->getDoctrine()->getManager();

        $entity = $em->getRepository('AcmeDemoBundle:Product')->find($id);

        if (!$entity) {
            throw $this->createNotFoundException('Unable to find Product entity.');
        }

        $deleteForm = $this->createDeleteForm($id);
        $editForm = $this->createForm(new ProductType(), $entity);
        $editForm->bind($request);

        if ($editForm->isValid()) {
            $em->persist($entity);
            $em->flush();

            return $this->redirect($this->generateUrl('product_edit', array('id' => $id)));
        }

        return array(
            'entity'      => $entity,
            'edit_form'   => $editForm->createView(),
            'delete_form' => $deleteForm->createView(),
        );
    }

has to be changed reduced into this:

    /**
     * Edits an existing Product entity.
     */
    public function editAction($id)
    {
        $em = $this->getDoctrine()->getManager();

        $entity = $em->getRepository('AcmeDemoBundle:Product')->find($id);

        if (!$entity) {
            throw $this->createNotFoundException('Unable to find Product entity.');
        }

        $editForm = $this->createForm(new ProductType(), $entity);
        $deleteForm = $this->createDeleteForm($id);

        $request = $this->getRequest();
        if ($request->getMethod() == 'POST') {

            $editForm->bind($request);

            if ($editForm->isValid()) {
                $em->persist($entity);
                $em->flush();

                return $this->redirect($this->generateUrl('edit', array('id' => $id)));
            }

        }

        return array(
            'entity' => $entity,
            'edit_form' => $editForm->createView(),
            'delete_form' => $deleteForm->createView(),
        );
    }

Relative Routing

As a bonus this bundle allows you to refer relatively to another route, so instead of writing in the view:

<!-- Acme/DemoBundle/Resources/views/Product/index.html.twig -->
<a href="{{ path('acme_demo.product.show', { 'id': entity.id }) }}">show</a>

You can simply write this:

<!-- Acme/DemoBundle/Resources/views/Product/index.html.twig -->
<a href="{{ path('show', { 'id': entity.id }) }}">show</a>

Or instead of writing this in the controller:

$this->generateUrl('acme_demo.product.show', array('id' => $id))

You can simply write this:

$this->generateUrl('show', array('id' => $id))

Since routes are now expected to be “{bundle}.{controller}.{action}” format and the current route is “acme_demo.product.index” the undefined route “show” can be automatically matched to “acme_demo.product.show”. Apart from enforcing consistency this also allows for greater reuse of (parts of) code.

Get the bundle

Now quickly go grab a copy of the LswDefaultRoutingBundle and enjoy its benefits!

 

Symfony2 CRUD generator

I have ran into a real hidden gem. A YouTube user called “goyocode” has put an excellent series on using the CRUD generator in Symfony2 online. The links are below:

0. How to Generate an Entity in Symfony2 (11:41)
http://www.youtube.com/watch?v=0SE09AlJSEI

1. How to Generate a Repository in Symfony2 (1:20)
http://www.youtube.com/watch?v=asC9LL4cHrs

2. How to Generate CRUD for an Entity in Symfony2 (4:36)
http://www.youtube.com/watch?v=Jqp781knyPM

3. How to Install and Configure the Sonata Admin Bundle (11:19)
http://www.youtube.com/watch?v=_ZGRs4Rd94A

4. How to Create a ManyToMany Relationship with Doctrine2 & Symfony2 (27:35)
http://www.youtube.com/watch?v=akV4D7cq4UQ

5. Hack to Update a ManyToMany Relationship Through the Inverse Side in Doctrine2  (4:00)
http://www.youtube.com/watch?v=kPrgoe3Jrjw

It clearly explains the process of generating applications using the (Doctrine) CRUD generator in video’s 0,1 and 2. In video 3 the (Sonata) admin generator is explained.  Video 4 and 5 explain the more advanced entity relationships “ManyToMany” and “inversed ManyToMany”.