Search This Blog

Wednesday, March 27, 2013

Symfony Project sfservermon Post 7 - Update Details - UpdaterServer class


Post 6 discussed the high level ServerUpdate class, this post finally documents the details of the UpdaterStatus class.

UpdaterStatus Class

This class in particular has about 90% of its code excellent taken from the original phpServerMon:

The class has the following methods:

  • Constructor - which saves references to the Entity manager, configuration and logger
  • getStatus - which calls update to update the server status.
  • update - which switches based on server type service or website to get the server status. It will recall itself recursively - to max runs to retry a connetion attempt.
  • updateService - checks a service server type
  • updateWebsite - checks a website server type.
  • notify - sends the required notification.
The first 3 methods are straightforward:
public function __construct($em, $configuration, $monitorLog)
{
    $this->em = $em;
    $this->config = $configuration;
    $this->monitorLog = $monitorLog;
}

public function setServer($server, $status_org) {
$this->clearResults();
$this->server = $server;
$this->status_org = $status_org;
}

/**
 * Get the new status of the selected server.
 *  If the update has not been performed yet it will do so first
 *
 * @return string
 */
public function getStatus() {

if(!$this->server) {
return false;
}
if(!$this->status_new) {
$this->update(3);
}
return $this->status_new;
}

public function update($max_runs=2)
{
switch($this->server->getType()) {
case 'service':
$result = $this->updateService($max_runs);
break;
case 'website':
$result = $this->updateWebsite($max_runs);
break;
}
return $result;
}



update Service Method

The updateService method checks a service -for example a database port on a server.
It requires a server tcp/udp port number and an ip address/hostname.
This method - and updateWebsite also record the response time.


protected function updateService($max_runs, $run = 1) {
// save response time
$time = explode(' ', microtime());
$starttime = $time[1] + $time[0];

@$fp = fsockopen ($this->server->getIp(), $this->server->getPort(), $errno, $errstr, 10);
Try to open a socket connection to the port using fsockopen , the last parameter is a timeout value - this should probably be stored in the server configuration, here it is 10 seconds.

$time = explode(" ", microtime());
$endtime = $time[1] + $time[0];
$this->rtime = ($endtime - $starttime);


$this->status_new = ($fp === false) ? 'off' : 'on';
$this->error = $errstr;
// add the error to the server array for when parsing the messages
$this->server->setError($this->error);
Was the socket opened? Check and save the setting, also save the error string.

@fclose($fp);

// check if server is available and rerun if asked.
if($this->status_new == 'off' && $run < $max_runs) {
return $this->updateService($max_runs, $run + 1);

If failed call again. return $this->status_new;

}

update Website Method.

This uses curl to opena connection to the website, it then checks the http status code.
An error can happen if
  • There is no response.
  • An unreadable response
  • Code is not as expected: HTTP/1.1 200 OK

protected function updateWebsite($max_runs, $run = 1) {
// save response time
$time = explode(' ', microtime());
$starttime = $time[1] + $time[0];

as before, record the start time.

$ch = curl_init();
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_URL, $this->server->getIp());
curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt ($ch, CURLOPT_TIMEOUT, 10);
curl_setopt ($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11');

// We're only interested in the header, because that should tell us plenty!
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
Create and initialise a curl object, we are only interested in the headers.

$headers = curl_exec ($ch);
curl_close ($ch);
Attempt to connect, and save the headers.
$time = explode(" ", microtime());
$endtime = $time[1] + $time[0];
$this->rtime = ($endtime - $starttime);
Record the time taken.
// the first line would be the status code..
$status_code = strtok($headers, "\r\n");
// keep it general
// $code[1][0] = status code
// $code[2][0] = name of status code
preg_match_all("/[A-Z]{2,5}\/\d\.\d\s(\d{3})\s(.*)/", $status_code, $code_matches);

Check and examine the http status.

if(empty($code_matches[0])) {
// somehow we dont have a proper response.
$this->error = 'no response from server';
$this->server->setError($this->error);
$this->status_new = 'off';
} else {
$code = $code_matches[1][0];
$msg = $code_matches[2][0];

// All status codes starting with a 4 or higher mean trouble!
if(substr($code, 0, 1) >= '4') {
$this->error = $code . ' ' . $msg;
$this->server->setError($this->error);
$this->status_new = 'off';
} else {
$this->status_new = 'on';
}
}

// check if server is available and rerun if asked.
if($this->status_new == 'off' && $run < $max_runs) {
return $this->updateWebsite($max_runs, $run + 1);
}


Recall if necessary

return $this->status_new;
}


The next post discussed the notification after a server status check.


Symfony Project sfservermon Post 6 - Update server status.

Updating the Server status.

The previous post allow us to create a new updater class and run it from the command line:


[johnr@sentos4 webserver]$ app/console servermon:update

It also talked about the initialisation of the ServerUpdate class using dependency injection as a Symfony service.

This post documents the actual update process, and involves 2 classes:
  • ServerUpdate - which uses the UpdaterStatus class to check the status for all required servers.
  • UpdaterStatus - updates a status for one particular server, it also provides a notify method to do the required notification.
At first thought much of the ServerUpdate code could be included in the UpdateCommand class - however the Symfony documentation for the console class reccommends that the command class do very little other than check any required parameters - for this application at least this is a very good reason - as an update can be initiated from the update action in the default Controller.

Much of this code is a straight port (read copy and paste) from the excellent code in phpServerMon application.

ServerUpdate Class.

This class has an update method: called from the updateCommand or the updateAction:


$serverUpdate =  $this->container = 
     $this->getApplication()->getKernel()->getContainer()->get('server_update');
$serverUpdate->update($serverLabel);

It does the following:
  1. Creates an array $servers[] of the required servers to update
  2. Reads the application configuration.
  3. Creates a Monitor log class.
  4. Creates a new updaterStatus class.
  5. Loops through each server: 
Reading the Required Servers:
If a server label is specified read that one or read all active servers:

if ($serverLabel) {
$servers = $this->em->getRepository('JMPRServerMonBundle:MonitorServers')->findByLabel($serverLabel);
} else {
$servers = $this->em->getRepository('JMPRServerMonBundle:MonitorServers')->findByActive('yes');
}

Getting the Configuration and Creating the required Classes.


//get Configuration.
$configuration = $this->em->getRepository('JMPRServerMonBundle:MonitorConfig')->getConfig();

$monitorLog = $this->em->getRepository('JMPRServerMonBundle:MonitorLog');

$updater = new UpdaterStatus($this->em, $configuration, $monitorLog);


The configuration data requires helper class in its repository: - getConfig.
The configuration is stored (persisted) as a series of database records - 1 record for each configuration setting, but is used throughout the application as an associative array:

public function getConfig() {
$configArray=array();
$configs = $this->findAll();
foreach($configs as $config) {
$configArray[$config->getKey()] = $config->getValue();
}
return $configArray;
}

Main Server loop:

This is where the work happens:
  1. Saving the old status
  2. Calling UpdateStatus to updating it
  3. Notifying and
  4. Updating the server entity


(1) $status_org = $server->getStatus();

// remove the old status from the array to avoid confusion between the new and old status
$server->setStatus('unknown');

(2) $updater->setServer($server, $status_org);

//check server status
$status_new = $updater->getStatus();

(3) //notify the nerds of applicable
$updater->notify();

//update server status
(4) $server->setLastCheck(new \DateTime('now'));
$server->setStatus($status_new);
$server->setError($updater->getError());
$server->setRtime($updater->getRtime());

if ($status_new = 'on') {
$server->setLastOnline($server->getLastCheck());
}
$this->em->persist($server);
$this->em->flush();

The next post goes into details on the Updater Status class.


Symfony Project sfservermon Post 5 Command Line Operation

This post covers the servers status update process, specifically the operation from the command line.
The previous post discussed managing - (creating, updating and deleting)  the server entities; we are now at the business end - the monitoring of the server status - which can be migrated to Symfony.
The first area to be covered is the access by command line.
The server status is designed to be updated regularly using a cron task. Which requires a command line interface  (CLI).


*/5 * * * * /usr/bin/php /home/phpservermon/webserver/cron/status.cron.php > /var/log/servermon.log 2>&1


The original phpservermon class - status.cron.php is a standard php script application that:

  • Includes config.php
  • Queries the database for active servers
  • Creates a new updater class
  • Loops through each of the servers
  • Assigns the server details and current status to the updater class.
  • Calls the updater getStatus() method to get the current status.
  • Calls the updater notify() method to handle (i.e. send required notifications) of the current status
  • Save the status to the database.

Command Class

This post covers the creation of the updater class and the evocation from the command line.
The symfony console component is used, this post documents the setup and use - in the Acme Demo bundle, in particular 

Command class.


class UpdateCommand extends Command
{
    protected function configure()
    {
        $this
            ->setName('servermon:update')
            ->setDescription('Update server status')
            ->addArgument(
                'serverLabel',
                InputArgument::OPTIONAL,
                'Which server to update?'
            )
        ;
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {

$this->container = $this->getApplication()->getKernel()->getContainer();

        $serverLabel= $input->getArgument('serverLabel');

$serverUpdate =  $this->container = $this->getApplication()->getKernel()->getContainer()->get('server_update');

$serverUpdate->update($serverLabel);

    }
}


This is similar to the hello world example - where it takes one optional argument - the server label. It does instantiate a serverUpdate class and calls with any argument (if supplied).

When run from the command line you see:


$ app/console --list
Symfony version 2.1.8-DEV - app/dev/debug
...
servermon
  servermon: update                     Update server status

It is located in a subfolder called Command in the bundle, and the class - ends with Command.
As suggested in the Symfony documentation the command class does little other than check the arguments and calls a dedicated class to perform the task. This is useful as the server update task can also be called from a controller.

Note: the optional argument - the srervername is an addition to the phpservermon original code, I added it when writing the previous post and have kept it as it is useful for testing.

Update (Service) Class.

The Update class - is a perfect candidate for a Symfony service. It needs a connection to the ORM, which is configured in the service file:
resources/config/services.yml


services:
  server_update:
    class: JMPR\ServerMonBundle\Update\ServerUpdate
    arguments: [ @doctrine.orm.entity_manager ]

Dependency injection is used to configure the ORM connection. Fabian has a great introduction blog post on dependency injection (from 2009 but still worth a read). Passed to the class - is the ORM entity manager (@doctrine.orm.entity_manager)

An initial version of the class - which just queries the Server repository to find the matching servers and prints them. As a feature - a server specified on the command line will be checked even if it is not active.

namespace JMPR\ServerMonBundle\Update;

use JMPR\ServerMonBundle\Entity;
use Doctrine\ORM\Mapping as ORM;
class ServerUpdate
{
  protected $em;

  public function __construct($em)
  {
        $this->em = $em;
  }


  protected function configure()
  {
  }

  public function update($serverLabel)
  {

if ($serverLabel) {
  $servers = $this->em->getRepository('JMPRServerMonBundle:MonitorServers')->findByLabel($serverLabel);
     echo "update: <". $serverLabel ."> " . count($servers) ." servers matched\n---";
} else {
  $servers = $this->em->getRepository('JMPRServerMonBundle:MonitorServers')->findByActive('yes');
  echo "update: " . count($servers) ." servers matched\n---";
}

foreach ($servers as $server) {
  echo "server: ". $server->getLabel() . ":" . $server->getIp() . "\n---";
}

    }

It is interesting at this point to compare this to the original phpservermon class: sfUpdate


# classes/sm/smUpdaterStatus.php

class smUpdaterStatus extends smCore {
}

The smCore class, is purely concerned with instantiating a link to the database - the same functionality provided by the Symfony service container:

abstract class smCore {
public $db;

function __construct() {
// add database handler
$this->db = $GLOBALS['db'];
}
}

So the original smCore class was designed to solve the same problem that the dependency injection addresses.
A final note on this topic - the updater class is called from a controller in response to an update request, the route was added:

server_mon_update:
    pattern:  /update
    defaults: { _controller: JMPRServerMonBundle:Default:update }
The base.html.twig template edited:
          <li><a href="{{ path('server_mon_update')}}">update</a></li>
and the controller action:
public function updateAction()
{
$updater = $this->get('server_update');
        $updater->update('');

        return $this->redirect($this->generateUrl('server_mon_homepage'));
}
The get method to the service container will construct an updater class and initialise it.

The next post discusses the ServerUpdate and UpdaterStatus classes in detail.




Monday, March 11, 2013

Symfony 2 - Customising Form Widget Display.

As you would expect with Symfony the layout and display for form widgets can be customised as needed.
I had a contact form with a radio buttons generated by this code:


$form = $this->createFormBuilder($contact)
->add('contactType', 'choice', array('label'=>'Select a Choice','expanded'=>true,'choices' => array('quote'=>'Request a Quote','support'=>'Technical Support','custservice'=>'Customer Service')))
->add('firstName', 'text', array('label'=>'first'))
->add('lastName', 'text', array('label'=>'last'))
->add('emailAddress', 'email', array('label'=>'Email'))
      ->add('phoneAreaCode', 'number', array('label'=>'###', 'max_length'=>3))
      ->add('phone1', 'number', array('label'=>'###', 'max_length'=>3))
      ->add('phone2', 'number', array('label'=>'####', 'max_length'=>4))
      ->add('message', 'textarea', array('required'=>false))
      ->getForm();
And rendered in a template:
<div class="TabbedPanelsContent">
  <div style="padding-left:40px; padding-right:150px; height:600px">
  <h2>Contact ACO</h2>
  <p>How can we help you?</p>


  <div class="ContactFormArea"> Select a Choice:
<form action="{{ path('acocusa_website_contactform') }}" method="post" {{ form_enctype(form) }}>
      <p>
    {{ form_widget(form.contactType) }}

      </p>
      <div class="ContactFields" style="width:580px; height:55px"> Name<br />
        <div class="ContactFieldElements">
          {{ form_widget(form.firstName) }}<!--<input name="FirstName" type="text" class="InputField" id="FirstName" size="22" />--><br /><label for="FirstName">First</label>
        </div>
        <div class="ContactFieldElements">
          {{ form_widget(form.lastName) }}<br /><label for="Surname">Surname</label>
        </div>
      </div>
   ....  
Which looks like this:

 However the designers wanted this:


In particular the Select a choice field needed to be displayed one option per line.
From the Symfony 2 cookbook - form_customization page I created a fragment to override the standard one. The originals are in vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Resources/views/Form
For the choice widget there are:
choice_widget.html.php
choice_widget_collapsed.html.php
choice_widget_expanded.html.php
choice_options.html.php
choice_widget_options.html.php

However these are php code not twig templates, I needed the original twig template code to modify.
A search on stackoverflow.com provided the answer, I added this code to the template:

{% block choice_widget_expanded %}
{% spaceless %}
<div {{ block('widget_container_attributes') }}>
{% for child in form %}
    <label class="radio">
        {{ form_widget(child, {'attr': {'class': attr.widget_class|default('')}}) }}
        {{ child.vars.label|trans({}, translation_domain) }}<br>
    </label>
{% endfor %}
</div>
{% endspaceless %}
{% endblock %}
Adding the <br> tags.
The spaceless tag specifies twig to remove all white page between html tags.





Symfony Project sfservermon - Post 3 First Pages.

This is post 3 in a series - back to previous post.
By the end the previous post we now have a set of database Entities and a database with sample data.
The next stage is to display it.

Servers Index Page.

Note, in a later post the edit pages have been generated by the generate:doctrine:crud tool. For this to work correctly the server_id database column needed to be mapped to an Entity variable called id, the Entity generator then generates a getId() method rather than getServerId() method or getConfigId() method.
So while the primary key in the database is server_id or config_id, in Symfony through Doctrine, it needs to be mapped as 'id'. This is done in the orm file:

JMPR\ServerMonBundle\Entity\MonitorServers:
  type: entity
  table: monitor_servers
  fields:
    id:
      id: true
      type: integer
      unsigned: false
      nullable: false
      column: server_id
      generator:
        strategy: IDENTITY

Various other fields had nullable set to true:  port, error, rtime, lastOnline, lastCheck.

Routing.

For the initial pages to display, these routes were added:


server_mon_homepage:
    pattern:  /
    defaults: { _controller: JMPRServerMonBundle:Default:index }

server_mon_log:
    pattern:  /log
    defaults: { _controller: JMPRServerMonBundle:Default:indexlog }



(Note in a later post these routes are edited to that the identifier complies with the Symfony Best Practices for Structuring Bundles
These are routes to the home (server list) page and the log page.

Default Controller.

A default controller was created with these methods:

    public function indexAction()
    {

$em = $this->getDoctrine()->getManager();
$servers = $em->getRepository('JMPRServerMonBundle:MonitorServers')->findAll();
        return $this->render('JMPRServerMonBundle:Default:servers.html.twig', array('title'=>'Servers','servers'=>$servers));
    }

    public function indexlogAction()
    {

$em = $this->getDoctrine()->getManager();

$statusLogs = $em->getRepository('JMPRServerMonBundle:MonitorLog')->findBy(array('type' => 'status'), array('datetime' => 'DESC'), 100);
$emailLogs = $em->getRepository('JMPRServerMonBundle:MonitorLog')->findBy(array('type' => 'email'), array('datetime' => 'DESC'), 100);
$smsLogs = $em->getRepository('JMPRServerMonBundle:MonitorLog')->findBy(array('type' => 'sms'), array('datetime' => 'DESC'), 100);

        return $this->render('JMPRServerMonBundle:Default:logs.html.twig'
        , array('title'=>'Logs', 'statusLogs'=>$statusLogs, 'emailLogs'=>$emailLogs, 'smsLogs'=>$smsLogs));
    }

Templates.

A base template - base.html.twig in app/Resources/views is created. The base template was created by
  1. Copying and pasting the html code from the phpServerMon projec
  2. Changing the css and javascript links - /css and /js, checking the image links (to '/img')
  3. Adding twig block tags for the title, stylesheets, pageContent and javascripts.
The base template:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    <title>{% block title %}PHP Server Monitor{% endblock %}</title>
  <!--CSS Files-->
<link type="text/css" href="/css/monitor.css" rel="stylesheet" />
<!--JavaScript Files-->
<script src="/js/monitor.js" type="text/javascript" ></script>

    {% block stylesheets %}{% endblock %}
    <link rel="icon" type="image/x-icon" href="{{ asset('favicon.ico') }}" />

</head>
<body>

  <div class="main">

    <div class="header">
      <div class="opensource"><a href="{{ path('server_mon_homepage')}}"><img src="/img/opensource.png" alt="Open Source" height="101px" /></a></div>
      <div class="menu">
        <h1>SERVER MONITOR</h1>
        <ul>
          <li><a href="{{ path('server_mon_homepage')}}" class="active">servers</a></li>
          <li><a href="index.php?type=users" class="">users</a></li>
          <li><a href="{{ path('server_mon_log')}}" class="">log</a></li>
          <li><a href="{{ path('server_mon_config')}}" class="">config</a></li>
          <li><a href="index.php?action=check">update</a></li>
          <li><a href="http://phpservermon.sourceforge.net" target="_blank">help</a></li>
        </ul>
      </div>
    </div>

    <div class="container">
    <strong>Current time: 12:52:17 PM</strong><br>

      {% block pageContent %}{% endblock %}
</div>
      {% block javascripts %}{% endblock %}

    <div class="footer">
      Powered by <a href="http://phpservermon.sourceforge.net" target="_blank">PHP Server Monitor v2.0.1</a><br/>
      Running on: John-Reidys-MacBook-2.local, refresh at: 12:52:17 PM
    </div>
  </div>

  </body>
</html>

It contains a number of broken links - links that need to be replaced as the site is ported to Symfony.

Two templates for the server list and log list:  servers.html.twig and logs.html.twig were created in the bundle (directory: ../src/JMPR/ServerMonBundle/Resources/views/Default)
The servers template is:
{# src/JMPR/ServerMonBundle/Resources/views/Default/index.html.twig #}
{% extends '::base.html.twig' %}
{% block title %}{{title}}{% endblock %}
{% block header %}


{% endblock %}
{% block pageContent %}
<h2>servers</h2>
<div class="message"><a href="">Add new?</a></div><br/>
<table cellpadding="0" cellspacing="0">
  <tr class="odd">
    <td>&nbsp;</td>
    <td><b>Label</b></td>
    <td><b>Domain/IP</b></td>
    <td><b>Port</b></td>
    <td><b>Type</b></td>
    <td><b>Last check</b></td>
    <td><b>Response time</b></td>
    <td><b>Last online</b></td>
    <td><b>Monitoring</b></td>
    <td><b>Send Email</b></td>
    <td><b>Send SMS</b></td>
    <td><b>Action</b></td>
  </tr>
  <tr class="odd">
    <td colspan="12" class="message">&nbsp</td>
  </tr>
{% for server in servers %}
<tr class="{{ loop.index is odd ? 'odd' : 'even' }}">
 <td><img id="status_{{server.id}}" src="/img/on.png" alt="on" height="30px" title="" /></td>
 <td>{{server.label}}</td>
 <td><a href="{{path('server_mon_log', {'server_id':server.id})}}">{{server.ip}}</a></td>
 <td>{{server.port}}</td>
 <td>{{server.type}}</td>
 <td>{{server.lastOnline|date("d-m-Y H:i:s")}}</td>
 <td>{{server.rtime}} s</td>
 <td>{{server.lastCheck|date("d-m-Y H:i:s")}}</td>
 <td>yes</td>
 <td>no</td>
 <td>no</td>
 <td>
   <a href=""><img src="/img/edit.png" alt="edit" title="Edit Server" /></a>
   &nbsp;
   <a href="javascript:sm_delete('', 'servers');"><img src="/img/delete.png" alt="delete" title="Delete Server" /></a>
 </td>
</tr>

{% endfor %}
</table>
{% endblock %}


It populates the title, pagecontent and javascript blocks. note that the links to the add new server, edit and delete are all empty.
With the test data and the templates this page now looks like:


The logs page is similar the main difference is that the entries are displayed in 3 tabs for status changes/emails and sms messages sent.

The work done in this post is a great example of the power of the Symfony framework.

The next post discusses the edit forms.