Search This Blog

Wednesday, January 30, 2013

Installing PHP extensions.

When using php, it won't be long before you will need to install an extension.
An example is with Symfony2 which recommends APC and intl extensions.
In fact intl is required for the FOSUserBundle, if intl is not installed you will get the error: The locale resource bundle could not be loaded for locale "en"
when trying to edit a user profile (e.g. ../../user/user/1/edit)

Use pecl to install the intl extension and then add a:
extenions=intl.so
to /etc/php.ini.

When doing this you might get the following errors:
[root@spectest php-5.3.4]# pecl install intl
downloading intl-2.0.1.tgz ...
Starting to download intl-2.0.1.tgz (149,430 bytes)
.................................done: 149,430 bytes
package.xml version "2.1" is not supported, only 1.0 and 2.0 are supported.
Download of "pecl/intl" succeeded, but it is not a valid package archive
Error: cannot download "pecl/intl"
In this case, upgrade pear, you might need to enter:
 pear upgrade -f pear


Note - intl requires libicu. Either download it from site.icu-project.org, or through your package installed (e.g.: yum  - package libicu-devel), if using yum it will most likely require icu and libicu as prerequisites.

Really apply Magento Catalog rules

One of the things the regular cron jobs to is to reapply the catalog rules.
A symptom of the catalog rules not working is when the price of an item in a shopping cart doesn't have a sale price applied to it - was it does when viewing/searching for them from the catalog.

The cron entry in the config.xml file for the catalog in the rules should have:
    <crontab>
        <jobs>
            <catalog_product_index_price_reindex_all>
                <schedule><cron_expr>0 2 * * *</cron_expr></schedule>
                <run><model>catalog/product_indexer_price::reindexAll</model></run>
            </catalog_product_index_price_reindex_all>
        </jobs>
    </crontab>
</config>

This shows the catalog reindexAll method in product_indexer_price being called every hour.
If cron is working correctly this should be getting called.
I had an experience with a v.1.5.1.0 store where this wasn't happening and every so often the rules would not be applied.
The code below - which I added to a custom cron script applies the rules directly.


    /**
     * Apply all active catalog price rules
     */
try {
    Mage::getModel('catalogrule/rule')->applyAll();
    Mage::app()->removeCache('catalog_rules_dirty');
    echo  'The rules have been applied.';
    // added JMR 08-01-2013
    Mage::log("Cron apply rules run");
} catch (Exception $e) {
    echo 'Unable to apply rules.';
    Mage::log("Cron Unable to apply rules");
    Mage::printException($e);
}



Configuring Symfony 2 with a Cpanel hosted site.

This post by at weblincs is useful - symfony-framework-cPanel-shared-hosting, however it is for Symfony 1.x.
As it states is best to use symbolic links to avoid any custom httpd config file configuration causing problems with Cpanel.
The Symfony 2 configuration is even simpler (I believe this is related to the configuration from Symfony v1 to v2) I did the following:

  1. Upload your symfony application.
  2. Extract it into a new folder -eg  /home/user/symfony_site, on a standared cPanel site this will be the same level as the public_html folder.
  3. Remove or rename the public_html folder.
    mv public_html public_html_original
  4. Create a new symbolic link linking public_html to the symfony web folder:
    ln -s symfony_site/web public_html
  5. Check the folder permissions - apache will warn in the log file if there is group write permission:
    chmod g-w symfony_site/web
    chmod g-w public_html
  6. Restart Apache (see reference at etwiki.cpanel.net):
    /usr/local/cpanel/scripts/restartsrv  httpd 
  7. check the permissions of app/cache and app/logs, clear the symfony cache.
One tip - if not using a source code control, this query will list modified files since a timestamp file on the top directory.


find . -newer LAST_UPLOAD -type f -o -path ./app/cache -prune -type f



Tuesday, January 22, 2013

Unix/Linux Shell Scripting

Test Mode Scripts.

If you want to have shell scripts that can run in a test mode - with more output or a different processing result, you can have a symbolic link to you existing script:
ln -s emailCheck.sh emailCheck_test.sh

An add to the script:
if [ `expr $0 : '.*test\.sh'` -gt 0 ]; then
  testMode=1
  testModeStr=test
else
  testMode=0
  testModeStr=
fi


Awk.

Awk (also known as gawk) predates perl and many other scripting languagues.
For more information see: www.gnu.org gawk manual
It is a stream processor - and can be considered a halfway house between sed and perl.
An example:


cat wstest2 | awk -F '\' '{printf "%s\\%s\\%s\\%s\\%s\\s\\%s\\%s echo %s >>\\temp\\wsarchivecheck\n", $1, $2, $3, $4, $5, $6, $7, $7;}'

This generates a dos shell script based on the source input.

Another - similar example, extract portion of the dos file name:

 cat t1|awk -F '\' '{ printf "%s\\%s\\%s\\%s\\%s %s\n", $1, $2, $3,$4,$5, gensub(/^([0-9]*)v.*/, "\\1", "g", $5);}'
for this input:
G:\mm4file09\105913\0000002\112693514v1.doc
G:\mm4file09\105913\0000002\112710470v1.DOC
G:\mm4file09\105913\0000002\112731114v1.DOC
it will generate this:
G:\mm4file09\105913\0000002\112693514v1.doc 112693514
G:\mm4file09\105913\0000002\112710470v1.DOC 112710470
G:\mm4file09\105913\0000002\112731114v1.DOC 112731114

gensub is a general purpose substitution function.

Generation of Awk Scripts.

Awk can be used as a pattern matcher similar to grep.
What is can also do is a multiple match and extraction.
For example, if you have a large webserver access log and want to do a mutliway split based on a path:
60.242.60.172 - - [27/Nov/2012:16:16:53 +1100] "GET /company/premium.htm HTTP/1.1" 200 15089 "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; GTB7.4; .NET4.0C; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 1.1.4322; BRI/2)"
66.249.74.152 - - [27/Nov/2012:16:18:14 +1100] "GET /company/zipindustries.htm HTTP/1.1" 404 307 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.74.4 - - [27/Nov/2012:16:18:15 +1100] "GET /company/zipindustries.htm HTTP/1.1" 404 309 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

This awk script will do this:
BEGIN {  }
$7 ~ /premium/ { print $0 >> "cologs/premium.log" }
$7 ~ /zipindustries/ { print $0 >> "cologs/zipindustries.log" }
$7 ~ /aaf/ { print $0 >> "cologs/aaf.log" }
END { }

If a source file lists the web sites to extact:
3ablinds  3a Blinds
premium    Premium Carpets
zipindustries Zip Industries


This script will generate the awk extraction file:
echo "BEGIN {  }" > extract.awk
while read shortname sitename; do
  #echo "Building awk script $shortname: $sitename"
  echo "\$7 ~ /${shortname}/ { print \$0 >> \"cologs/${shortname}.log\" }" \
    >> extract.awk
done < $liblisthome/local.pages
echo "END { }" >> extract.awk

To run it:
awk -f extract.awk /var/log/httpd/access_log

Friday, January 18, 2013

Symfony 2 - Admin Generator - Sonata Admin Bundle

the Sonata Admin bundle -was originally written for Symfony v2.0, I have been attempting to get it installed and working for v2.1

Installation.

There is updated documentation at https://github.com/sonata-project/SonataAdminBundle/pull/814/files.
To summarise:
php composer.phar  require sonata-project/admin-bundle dev-master
php composer.phar update

You will probably need to set to minium stability to dev, not the dev-master version constraint.
I also changed the requirement for the doctrine-bundle to 1.* from 1.1.*

If you have problems updating particular components - e.g. assetic or Trig-extensions (get an error: fatal: No such remote 'composer' message) remove that item from the vendors directory.

Add to app/AppKernel.php:
        new Sonata\AdminBundle\SonataAdminBundle(),
        new Sonata\BlockBundle\SonataBlockBundle(),
        new Sonata\jQueryBundle\SonatajQueryBundle(),
        new Knp\Bundle\MenuBundle\KnpMenuBundle(),

and app/config/config.yml:
sonata_admin:
    title: Jobeet Admin

sonata_block:
    default_contexts: [cms]
    blocks:
        sonata.admin.block.admin_list:
            contexts:   [admin]

        sonata.block.service.text:
        sonata.block.service.action:
        sonata.block.service.rss:

Configure routing in app/config/routing.yml:
admin:
    resource: '@SonataAdminBundle/Resources/config/routing/sonata_admin.xml'
    prefix: /admin

_sonata_admin:
    resource: .
    type: sonata_admin
    prefix: /admin

I also uncommented the translator item under framework as per the documentation. You can then browse to:

http://<app-url>/app_dev.php/admin/dashboard

If the images (and css) don't appear, install the assets:
php app/console assets:install web

Doctrine Support.

Sonata is ORM independent, to install support for Doctrine, add:
php composer.phar require sonata-project/doctrine-orm-admin-bundle  dev-master 

Configuration Administration.


Step 1 Link the Required Models.

Create CRUD controllers for the models you want to have administered. These extend the Sonata CRUD controllers, these map the model and adminstration sections (forms,list,show) for ProductRange:
// src/Test/WebsiteBundle/Controller/ProductRangeAdminController.php
namespace Test\WebsiteBundle\Controller;

use Sonata\AdminBundle\Controller\CRUDController as Controller;

class ProductRangeAdminController extends Controller
{
}

Step 2 Confgure Services.

Configure services in Resourse/config/services.yml
    test.websitebundle.admin.productrange:
        class: Test\WebsiteBundle\Admin\ProductRangeAdmin
        tags:
            - { name: sonata.admin, manager_type: orm, group: website, label: ProductRange }
        arguments: [null, Test\WebsiteBundle\Entity\ProductRange, TestWebsiteBundle:ProductRangeAdmin]

Now when you browse to http://<app-url>/app_dev.php/admin/dashboard, you see the classes you have mapped - togther with CRUD controls and row counts, however the forms are empty.

Step 3 Configure the CRUD Forms.

To show the required fields, edit the admin classes, for ProductRange:
class ProductRangeAdmin extends Admin
{
 
 // setup the default sort column and order
  protected $datagridValues = array(
      '_sort_order' => 'ASC',
      '_sort_by' => 'name'
  );
    
  protected function configureFormFields(FormMapper $formMapper)
  {
      $formMapper
          ->add('name')
          ->add('title')
          ->add('slug')
          ->add('description')
          ->add('shortDescription')
      ;
  }

  protected function configureDatagridFilters(DatagridMapper $datagridMapper)
  {
      $datagridMapper
          ->add('name')
          ->add('title')
      ;
  }

  protected function configureListFields(ListMapper $listMapper)
  {
      $listMapper
          ->addIdentifier('name')
          ->add('title')
      ;
  }

Adding User Authentication.

User authentication is done using the FriendsOfSymfony UserBundle The steps are:
Install bundle as normal:
./composer.phar require friendsofsymfony/user-bundle
./composer.phar require sonata/user-bundle
./composer.phar require sonata/easy-extends-bundle

load in app/AppKernel.php
           new FOS\UserBundle\FOSUserBundle(),
           new Sonata\UserBundle\SonataUserBundle('FOSUserBundle'),
           new Sonata\EasyExtendsBundle\SonataEasyExtendsBundle(),
Configure in config.yml, note reference to user entity in the Sonata Userbundle:
fos_user:
    db_driver: orm
    firewall_name: main
    user_class: Application\Sonata\UserBundle\Entity\User
Add routes:
fos_user_security:
    resource: "@FOSUserBundle/Resources/config/routing/security.xml"
 
fos_user_profile:
    resource: "@FOSUserBundle/Resources/config/routing/profile.xml"
    prefix: /profile
 
fos_user_register:
    resource: "@FOSUserBundle/Resources/config/routing/registration.xml"
    prefix: /register
 
fos_user_resetting:
    resource: "@FOSUserBundle/Resources/config/routing/resetting.xml"
    prefix: /resetting
 
fos_user_change_password:
    resource: "@FOSUserBundle/Resources/config/routing/change_password.xml"
    prefix: /change-password

soanata_user:
    resource: '@SonataUserBundle/Resources/config/routing/admin_security.xml'
    prefix: /admin
Use the extends bundle to create a user bundle
php app/console sonata:easy-extends:generate SonataUserBundle

This creates a new user bundle under app/Application
Which needs to be added to app/AppKernel.php:

           new Application\Sonata\UserBundle\ApplicationSonataUserBundle(),

I moved this from app to src.


Configure security:
Section 1 - classes and providers

security:
    encoders:
        FOS\UserBundle\Model\UserInterface: sha512
    role_hierarchy:
        ROLE_ADMIN:       ROLE_USER
        ROLE_SUPER_ADMIN: [ROLE_USER, ROLE_SONATA_ADMIN, ROLE_ADMIN, ROLE_ALLOWED_TO_SWITCH]
        SONATA:
            - ROLE_SONATA_PAGE_ADMIN_PAGE_EDIT  # if you are not using acl then this line must be uncommented
    providers:
        fos_userbundle:
            id: fos_user.user_manager

Section 2 - firewalls:

    firewalls:
        # -> custom firewall for the admin area of the URL
        admin:
            pattern:      /admin(.*)
            form_login:
                provider:       fos_userbundle
                login_path:     /admin/login
                use_forward:    false
                check_path:     /admin/login_check
                failure_path:   null
            logout:
                path:           /admin/logout
            anonymous:    true
        # -> end custom configuration
        # defaut login area for standard users
        main:
            pattern:      .*
            form_login:
                provider:       fos_userbundle
                login_path:     /login
                use_forward:    false
                check_path:     /login_check
                failure_path:   null
            logout:       true
            anonymous:    true

Section 3 - acl:

   # URL of FOSUserBundle which need to be available to anonymous users
        - { path: ^/_wdt, role: IS_AUTHENTICATED_ANONYMOUSLY }
        - { path: ^/_profiler, role: IS_AUTHENTICATED_ANONYMOUSLY }
        - { path: ^/login$, role: IS_AUTHENTICATED_ANONYMOUSLY }
        # -> custom access control for the admin area of the URL
        - { path: ^/admin/login$, role: IS_AUTHENTICATED_ANONYMOUSLY }
        - { path: ^/admin/logout$, role: IS_AUTHENTICATED_ANONYMOUSLY }
        - { path: ^/admin/login-check$, role: IS_AUTHENTICATED_ANONYMOUSLY }
        # -> end
        - { path: ^/register, role: IS_AUTHENTICATED_ANONYMOUSLY }
        - { path: ^/resetting, role: IS_AUTHENTICATED_ANONYMOUSLY }
        # Secured part of the site
        # This config requires being logged for the whole site and having the admin role for the admin part.
        # Change these rules to adapt them to your needs
        - { path: ^/admin, role: [ROLE_ADMIN, ROLE_SONATA_ADMIN] }
        - { path: ^/.*, role: IS_AUTHENTICATED_ANONYMOUSLY }

Update the schema:
php app/console doctrine:schema:update --force


Note: if you get the error:

[Doctrine\DBAL\DBALException]                                                               
  Unknown column type "json" requested.
Add to the config.yml:


doctrine:
    dbal:
        ...
        types:
            json: Sonata\Doctrine\Types\JsonType


You might need to install the sonata doctrine extensions:

        "sonata-project/doctrine-extensions": "dev-master",

Also note, the schema update command should return a status similar to:
Updating database schema...
Database schema updated successfully! "5" queries were executed

If it doesn't, check the ApplicationSonataUserBundle is loaded in AppKernel.
If the bundle is getting loaded check if auto_mapping is configured for the ORM, either:
Short form:
    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
        auto_mapping: true

Longer form:
    orm:
        auto_generate_proxy_classes: "%kernel.debug%"

        default_entity_manager: default
        entity_managers:
          default:
            auto_mapping: true
            mappings:
              ACOCUSAWebsiteBundle: ~





Final Steps.

clear the cache and install any missing assets:

php app/console assets:install web
php app/console cache:clear


Then you can add 2 users, a 'super-admin' and a regular user:

php app/console fos:user:create admin admin@example.com password --super-admin

php app/console fos:user:create testuser test@example.com password


Final Step, check translation configuration and add form text:
In app/config/config.yml, uncomment and edit:
    translator:      { fallback: "en" }

Create the file: ../Application/Sonata/UserBundle/Resources/translations/SonataUserBundle.en.yml
# app/Application/Sonata/UserBundle/Recources/translations/SonataUserBundle.en.yml
form:
  label_username:             Username
  label_email:                Email
  label_plain_password:       Password (Plain)
  label_groups:               Groups
  label_roles:                Roles
  label_locked:               Locked
  label_expired:              Expired
  label_enabled:              Enabled
  label_credentials_expired:  Credentials Expired
  label_name:                 Name
list:
  label_username:             Username
  label_email:                Email
  label_enabled:              Enabled
  label_locked:               Locked
  label_created_at:           Created At
  label_roles:                Roles
  label_name:                 Name
filter:
  label_username:             Username
  label_locked:               Locked
  label_email:                Email
  label_id:                   ID
  label_name:                 Name

If you get an error similar to: The locale resource bundle could not be loaded for locale "en", check you have done the translation settings in config.yml.
Also check that you have the intl php extension installed.

The symfony config.php script will warn if you have an older version of the icu libraries. On a Centos 5.4 server, the version installed by yum was version 3.0. I downloaded and installed v50.1site.icu-project.org download page icu library from


Further Customisation.

Once running you are likely to want to customise the admin forms.
This is done by editing and Admin/<Entity>Admin.php classes.
This page has some more information. In particular Sonata will base the field type on the Doctrine column type.
Also by default all fields are assumed as required (ie. required==true).
To override a field, for example to set a field to be a true/false field, edit the configFormFields method:

protected function configureFormFields(FormMapper $formMapper) { $formMapper ->add('partNo') ->add('productGroup') ->add('description') ->add('length') ->add('slotSize') ->add('intakeArea') ->add('lockingType') ->add('enabled', 'sonata_type_boolean') ; }

This is then handled by the class ./vendor/sonata-project/admin-bundle/Sonata/AdminBundle/Form/Type/BooleanType.php; looking at the source for this you can see that
    const TYPE_YES = 1;

    const TYPE_NO = 2;

Which may not be what you expect.

Adding __toString Methods.


Also you are likely to want to added __toString methods to each of the classes handled by the Admin bundle. This is needed for display when editing a object and to enable assignment of child/parent mapping through a drop down menu.
For example:

    /**
     * Get string value
     *
     * @return string
     */
    public function __toString()
    {
        return $this->title;
    }