Sphinx Search: first step to “fail save” queries and automatic load balancing

I am very excited about new Sphinx features which added in 2.1.1-beta.
One of the feature is agent mirrors which can be used for fail-over queries and load balancing.

In this short guide I will describe how in 3 easy steps make “fail save” queries in Sphinx Search.

1. Download latest sphinx version from the trunk

svn checkout http://sphinxsearch.googlecode.com/svn

In Sphinx Search 2.1.1 were added new syntax which lets you define so-called agent mirrors that can be used interchangeably when processing a search query. Master server keeps track of mirror status (alive or dead) and response times, and does automatic fail-over and load balancing based on that.
Read More »

Posted in Sphinx Search | Tagged | Leave a comment

I am at github – contributing to open source and sphinx search modules

Follow me and our open source project at Github.
If your are interesting in Sphinx Search and want to contribute to Open Source then following projects are the best choice to start:

Sooner we will add more Sphinx Search related projects, so stay in touch.

Posted in Open Source, Sphinx Search | Tagged , , | Leave a comment

High performance BuildExcerpts() with Sphinx Search

I published article at Ivinco.com blog about Sphinx Search BuildExcerpt optimisation.

Since version 2.0.1 Sphinx has ability to build snippets in a parallel mode.
Sphinx parallel processing is controlled by ‘dist_threads’ option, which tells searchd how many CPUs should be utilized for search processing. This parameter is also used by BuildExcerpts() API call in combination with ‘load_files’ option.
These two options allow Sphinx to build excerpts in a parallel mode which works much faster for huge amount of texts for processing.

Read more.

Posted in Sphinx Search | Tagged , | Leave a comment

MVA support is now available in Real-Time indexes!

Sphinx Technologies announced MVA (Multi Value Attribute) support in Sphinx Real-Time indexes.

As promised we’re letting you know: MVA support is just committed to Sphinx code base!

If you’d like to be early adopter and help Sphinx community with testing – please feel free to download current trunk using:

svn checkout http://sphinxsearch.googlecode.com/svn/trunk

Posted in Sphinx Search | Tagged | Leave a comment

Auto load custom resource model in Zend Framework

In that mini how to I want to describe how to setup custom resource in Zend Framework.
My goals are:

  • Ability setup resource settings in application.ini
  • Support custom folder for resource models like folder ‘models’ for database resource
  • Auto load

Custom resource loader

First I need to create custom resource loader for my model, it will allow me to use ‘resources.mymodel’ namespace in application.ini.

This simple example illustrate the basics of custom application resource class:

class My_Resource_Mymodel extends Zend_Application_Resource_ResourceAbstract
{
   const DEFAULT_REGISTRY_KEY = 'Mymodel';
 
   protected $_model;
 
   public function init()
   {
      return $this->getModel();
   }
 
   public function getModel()
   {
       if (null == $this->_model){
           $options = $this->getOptions();
 
           $host = !empty($options['host']) ?  $options['host'] : '';           
 
           $this->_model = new My_Model($host);
 
           Zend_Registry::getInstance()->set('mymodel', $this->_model);
       }
 
       return $this->_model;
   }
}

Place the above code in the library/My/Resource/Mymodel.php

It’s time to setup application.ini.
By default you can find application.ini in application/configs/ folder.

; Add 'My' namespace to the autoloader
autoloadernamespaces[] = My
;setup path to the Resource class
pluginpaths.My_Resource = APPLICATION_PATH "/../library/My/Resource"
;Finally, setup resource setting 'host' for 'mymodel'. 
;I used host as an example, it is up to your which settings to use.
resources.mymodel.host = "127.0.0.1"

Setup custom folder for ‘Mymodel’ resource

It is very simple operation, but it was very hard to realize how to do it.
Thanks to Andrey Korolyov who help me realize how to do it.

Open Bootstrap.php which usually placed in application folder and add following code to initialize ‘mymodels’ folder as new resource type for Mymodel namespace:

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
    protected function _initResourceMymodel()
    {
        $moduleLoader = $this->getResourceLoader();        
        $moduleLoader->addResourceType('Mymodel', 'mymodels', 'Mymodel');
        return $moduleLoader;
    }   
}

Definition of ‘Mymodel’ classes could look like:

class Mymodels_Articles 
{
}

Place this code in the application/mymodels/Articles.php and enjoy!

I would be glad to hear from your is that article was helpful.
Thank your!

Posted in Development | Tagged , , | Leave a comment

Autoloading in Zend Framework: Namespace matching

In short: if the class namespace prefix is not in a list of registered namespaces, return FALSE immediately. This allows for more optimistic matching, as well as fallback to other autoloaders.

How it works?
By default ZF autoload only one namespace with prefix ‘Zend’. It allow to use any Zend_* classes in application out of the box.

But, what to do if you need autoload you own library of classes i.e. with prefix ‘My’.
There are several ways to do it:

1. You can register new namespace in application.ini:

autoloadernamespaces[] = My

2. You can register new namespace in Bootstrap function:

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
{
    public function _initAutoload()
    {
        $loader = Zend_Loader_Autoloader::getInstance();
        $loader->registerNamespace('My_');
        //or for multiple namespaces
        $loader->registerNamespace(array('Foo_', 'Bar_'));
    }
}

3. You can tell Zend_Loader_Autoloader to act as a “fallback” autoloader:

$loader->setFallbackAutoloader(true);

This method is not recommended to use.
Internally, Zend_Loader_Autoloader uses Zend_Loader::loadClass() to load classes. That method uses include() to attempt to load the given class file. include() will return a boolean FALSE if not successful — but also issues a PHP warning.

For the classes which don’t follow PEAR relationship with the filesystem, you can register you own autoload method:

$loader->pushAutoloader('my_autoloader', 'My_');

Append function ‘my_autoloader’ to the stack, to manage classes with the prefix ‘My_’.
Simple ‘my_autoloader’ function could look like:

function my_autoloader($class_name) {
    include 'My/classes/'.$class_name . '.php';
}
Posted in Development | Tagged , , , | Leave a comment

ExpressionEngine development: custom pagination

In this short example I will show how to create custom pagination links in ExpressionEngine for module or plugin.

Core module file:

function results()
{
        //get pagination parameters offset from the url and limit from the template tag
        $offset = (int)$this->EE->uri->segment(4);                
        if (empty($offset)){
            $offset = 0;
        }
        $limit = (int)$this->EE->TMPL->fetch_param('limit', 10);
 
        //skip the model logic.....
 
        //build pagination
        $this->EE->load->library('pagination');
        $config['base_url'] = $this->_getBaseUrl();
        $config['uri_segment'] = 4;
        $config['total_rows'] = $total_found;
        $config['per_page'] = $limit;
 
        $this->EE->pagination->initialize($config);
 
        $pagination = $this->EE->pagination->create_links();
 
        $tagdata = $this->EE->TMPL->parse_variables_row($this->EE->TMPL->tagdata, array('paginate' => $pagination));        
 
        return $tagdata;
}
function _getBaseUrl()
{
        $url = $this->EE->functions->create_url(
                $this->EE->uri->segment(1). '/' .
                $this->EE->uri->segment(2). '/' .
                $this->EE->uri->segment(3)
                );
        return $url;
}

Note: ‘base_url’ contain only first 3 segments of url, fourth segment used as offset number.

Template:

<exp:mysearch:results limit="10">
 
<!--skip presentation-->
 
{if count == total_results}
{paginate}
{/if}
 
</exp:mysearch:results>

Output will be:
1 2 3 >

Thank you!

Posted in Development, Projects | Tagged , , , | Leave a comment

Welcome to My blog

Welcome to My blog. The content of this blog will be concentrated around one main topic - my experience in Search Technologies, High Load web sites and Open Source.

I will be sharing advanced, high level techniques for building and scaling up web application, best implementations of Search Technologies like Sphinx Search and I will give my best advice how to push up an Open Source software.

Besides that, I will share a bit of my personal life and my living experience in Slovakia where I recently moved from my Motherland Ukraine.

I’ll be posting roughly once a week. Check back soon for the first guides.

Posted in Personal, Projects | 6 Comments