Fun ab test of PHP Laravel compare to GO lang

I recently read Building API with GO and I like it’s second part where author perform a load of PHP Laravel compared to Go lang.
As a junior Golang learner I repeat all steps except I did use Apache Benchmark for testing and Vagrant for hosting MySQL instance and PHP Laravel.
At the end application was consist of 1 api call which fetch 1 record from MySql database.

My load test setup was 20 concurrent users which hits application during 10 seconds:

Finished 10053 requests
Failed requests: 0
Time per request: 19.897 [ms] (mean)
Requests per second: 1005.17 [#/sec] (mean)

PHP Laravel

Finished 839 requests
Failed requests: 0
Time per request: 238.556 [ms] (mean)
Requests per second: 83.84 [#/sec] (mean)

Golang was 12 times faster, nevertheless both finished with zero errors.
Performance is another big plus for GO lang.
Have fun! :)

How to monitor Nginx logs with Splunk

As a DevOps guy I often do incident analysis, post deployment monitoring and usual logs checks. If you also is using Splunk as me when let me show for you few effective Splunk commands for Nginx logs monitoring.

Extract fileds

To make commands works Nginx log fields have to be extracted into variables.

Where are 2 ways to extract fields:

  1. By default Splunk recognise “access_combined” log format which is default format for Nginx. If it is your case congratulations nothing to do for you!
  2. For custom format of logs you will need to create regular expression. Splunk has built in user interface to extract fields or you can provide regular expression manually.


Website traffic over time and error rate

Unexpected spike in traffic or in error rate are always first thing to look for. Following command build a time chart with response codes. Codes 200/300 is your normal traffic and 400/500 is errors.

timechart count(status) span=1m by status

Website traffic in Splunk

Response time

How do you know if your website running slowly?
For response time I suggest to use 20, 85 and 95 percentile as metrics.
You also can think of average response time metric, but low average response time doesn’t show that website is OK, so I am not using that metric in the query.

timechart perc20(request_time), perc85(request_time), perc95(request_time) span=1m

Response time in Splunk

Traffic by IP

Checking which IPs are most popular is a good way to spot bad guys or misbehaving bot.

top limit=20 clientip

Traffic by IP with splunk

Top of error page

Looking for pages which produce most errors like 500 Internal Server Error or not found pages like 404? Following two queries give you exactly that information.

Top error pages

search status >= 500 | stats count(status) as cnt by uri, status | sort cnt desc

Top 40x error pages

search status >= 400 AND status < 500 | stats count(status) by uri, status | sort cnt desc

TOP nginx error urls with Splunk

Number of timeouts(>30s) per upstream

When you are using Nginx as a proxy server it is very useful to see if any of upstreams are getting timeouts.
Timeouts could be a symptom for: slow application performance, not enough system resources or just upstream server is down.

search upstream_response_time >= 30 | stats count(upstream_response_time) as upstreams by upstream

Splunk get timeout nginx upstreams

Most time consuming upstreams

Most time consuming upstreams showing which of servers are already overloaded by requests and giving you a hint when application needs to be scaled

stats sum(upstream_response_time), count(upstream) by upstream

Most time consuming upstreams

In conclusion

Splunk functions like timechart, stats and top is your best friends for data aggregation. They are like unix tools - the more tools you know the more easier is to build powerful commands.

DevOps blog

It’s already 1 year and a half since I pass to the new level and became full time DevOps. Since then I learn a lot about DevOps role in a company, best practices and tools. So, I feel like it is time when I can start to share the knowledge and help other to get into DevOps world.


Sphinx Search: first step to “fail save” queries and automatic load balancing

I am very excited about new Sphinx features which added in 2.1.1-beta.
One of the feature is agent mirrors which can be used for fail-over queries and load balancing.

In this short guide I will describe how in 3 easy steps make “fail save” queries in Sphinx Search.

1. Download latest sphinx version from the trunk

svn checkout

In Sphinx Search 2.1.1 were added new syntax which lets you define so-called agent mirrors that can be used interchangeably when processing a search query. Master server keeps track of mirror status (alive or dead) and response times, and does automatic fail-over and load balancing based on that.
Continue reading Sphinx Search: first step to “fail save” queries and automatic load balancing

High performance BuildExcerpts() with Sphinx Search

I published article at blog about Sphinx Search BuildExcerpt optimisation.

Since version 2.0.1 Sphinx has ability to build snippets in a parallel mode.
Sphinx parallel processing is controlled by ‘dist_threads’ option, which tells searchd how many CPUs should be utilized for search processing. This parameter is also used by BuildExcerpts() API call in combination with ‘load_files’ option.
These two options allow Sphinx to build excerpts in a parallel mode which works much faster for huge amount of texts for processing.

Read more.

MVA support is now available in Real-Time indexes!

Sphinx Technologies announced MVA (Multi Value Attribute) support in Sphinx Real-Time indexes.

As promised we’re letting you know: MVA support is just committed to Sphinx code base!

If you’d like to be early adopter and help Sphinx community with testing – please feel free to download current trunk using:

svn checkout

Auto load custom resource model in Zend Framework

In that mini how to I want to describe how to setup custom resource in Zend Framework.
My goals are:

  • Ability setup resource settings in application.ini
  • Support custom folder for resource models like folder ‘models’ for database resource
  • Auto load

Custom resource loader

First I need to create custom resource loader for my model, it will allow me to use ‘resources.mymodel’ namespace in application.ini.

This simple example illustrate the basics of custom application resource class:

class My_Resource_Mymodel extends Zend_Application_Resource_ResourceAbstract
   const DEFAULT_REGISTRY_KEY = 'Mymodel';
   protected $_model;
   public function init()
      return $this->getModel();
   public function getModel()
       if (null == $this->_model){
           $options = $this->getOptions();
           $host = !empty($options['host']) ?  $options['host'] : '';           
           $this->_model = new My_Model($host);
           Zend_Registry::getInstance()->set('mymodel', $this->_model);
       return $this->_model;

Place the above code in the library/My/Resource/Mymodel.php

It’s time to setup application.ini.
By default you can find application.ini in application/configs/ folder.

; Add 'My' namespace to the autoloader
autoloadernamespaces[] = My
;setup path to the Resource class
pluginpaths.My_Resource = APPLICATION_PATH "/../library/My/Resource"
;Finally, setup resource setting 'host' for 'mymodel'. 
;I used host as an example, it is up to your which settings to use. = ""

Setup custom folder for ‘Mymodel’ resource

It is very simple operation, but it was very hard to realize how to do it.
Thanks to Andrey Korolyov who help me realize how to do it.

Open Bootstrap.php which usually placed in application folder and add following code to initialize ‘mymodels’ folder as new resource type for Mymodel namespace:

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
    protected function _initResourceMymodel()
        $moduleLoader = $this->getResourceLoader();        
        $moduleLoader->addResourceType('Mymodel', 'mymodels', 'Mymodel');
        return $moduleLoader;

Definition of ‘Mymodel’ classes could look like:

class Mymodels_Articles 

Place this code in the application/mymodels/Articles.php and enjoy!

I would be glad to hear from your is that article was helpful.
Thank your!

Autoloading in Zend Framework: Namespace matching

In short: if the class namespace prefix is not in a list of registered namespaces, return FALSE immediately. This allows for more optimistic matching, as well as fallback to other autoloaders.

How it works?
By default ZF autoload only one namespace with prefix ‘Zend’. It allow to use any Zend_* classes in application out of the box.

But, what to do if you need autoload you own library of classes i.e. with prefix ‘My’.
There are several ways to do it:

1. You can register new namespace in application.ini:

autoloadernamespaces[] = My

2. You can register new namespace in Bootstrap function:

class Bootstrap extends Zend_Application_Bootstrap_Bootstrap
    public function _initAutoload()
        $loader = Zend_Loader_Autoloader::getInstance();
        //or for multiple namespaces
        $loader->registerNamespace(array('Foo_', 'Bar_'));

3. You can tell Zend_Loader_Autoloader to act as a “fallback” autoloader:


This method is not recommended to use.
Internally, Zend_Loader_Autoloader uses Zend_Loader::loadClass() to load classes. That method uses include() to attempt to load the given class file. include() will return a boolean FALSE if not successful — but also issues a PHP warning.

For the classes which don’t follow PEAR relationship with the filesystem, you can register you own autoload method:

$loader->pushAutoloader('my_autoloader', 'My_');

Append function ‘my_autoloader’ to the stack, to manage classes with the prefix ‘My_’.
Simple ‘my_autoloader’ function could look like:

function my_autoloader($class_name) {
    include 'My/classes/'.$class_name . '.php';