Our switch towards middleware ecosystems is a forward-thinking step for the PHP ecosystem
Purpose Of This Interview
This is the #39th set of PHP Interview to help aspiring PHP developers & PHP fans alike to get inspired by listening from those PHP guys who are already highly involved into the PHP Ocean and being ‘there’ taming the waves, surfing better than ever to make themselves an awesome PHP Expert both in their own eyes (for self-accomplishment) and for the PHP Community.
On the other side, this is an opportunity for new PHPers to get to know their “PHP Elders“. I hope you will derive as much fun to read my interviews as I’m having by interviewing those #PHPantastic guys.
A Small Intro..
For those that are new to this blog, back in October 28, 2012 I hosted Mr Matthew Weier O’Phinney (MWOP) for the very first time on 7PHP, for a… wait for it.. PHP interview of course!
At that time, I also tagged him as the “Sir Alex Ferguson of the Zend Framework family“. And as of today July 2018, i.e 6yrs after, I still stand by this sentence. Why?
Just like Sir Alex, Mr Matthew, commonly known as the Supreme Allied Commander of Zend Framework (a special honorary title given to him by The Don of The PHP Community, Mr Cal Evans) has and is doing an outstanding work behind the scene for both Zend Framework and PHP.
I was lucky enough (and eternally thankful to the PHP Community) to have been in a room of PHP experts, at SunshinePHP Conference – in Miami Feb 2015, to hear his wisdom during a PHP-fig group discussion on HTTP message interfaces – for PSR-7, as well as while I attended one of his talks.
I cannot say it enough – MWOP is an awesome human being and a fantastic PHP developer.
And Now The Interview…
It all started when I saw the following tweet from MWOP:
— Matthew Weier O’Phinney (@mwop) April 24, 2018
Before we progress, I am using the following throughout the rest of this article interview:
>> Hey Matthew! For those that might not know you, can you please briefly introduce yourself and your relationship with Zend/Rogue Wave?
I am a Principal Engineer at Zend Technologies, a RogueWave Company. I’ve worked with Zend since 2005, starting first in their (then nascent) eBiz division, and then transitioning full-time to the Zend Framework team in 2007, following several successful initiatives I led on the project while working in eBiz.
In 2008, I was promoted to Software Architect, in 2009 to Project Lead, and in 2013 was promoted to Principal Engineer.
I remain project lead for Zend Framework and its associated components and sub-projects (such as Apigility and Expressive) to this day, and the upkeep and development of them is my primary job responsibility.
I also am tasked with working on the development of standards with PHP-FIG, and reviewing incoming features for the PHP language to see how they might affect how we develop the projects.
>> Just so we are all clear, what are we meaning by “components”? Are these independent PHP libraries, 100% decoupled from ZF, that can be used outside the scope of any ZF-based application?
>> What was the latest version of PHP that these components supported, prior to the move to PHP v7.2?
>> I do not have the whole picture about all things happening at ZF – are you guys working directly into compatibility with PHP v7.2 without going through 7.0, 7.1 ?
For the task of providing PHP 7.2 compatibility, most components already do; we just haven’t provided proof via our continuous integration.
In a few cases, though, we’ve discovered we were using deprecated functionality and/or were defining signatures in class extensions that are no longer allowed starting in 7.2 (e.g., you cannot add or remove by-reference binding to arguments or a method on an extension). This latter change, becoming more strict with regards to signatures, has been particularly problematic and revealing about what both we and projects we depend on do.
In particular, PHP 7.2 became more strict with regards to how it allows extending classes to define optional parameters in methods. Prior to 7.2, you could change the default value in an extending class.
For example, if you had defined:
public function get($name, array $options = null)
Then an extending class could override the method and define it as follows:
public function get($name, array $options = )
>> I want to understand the process involved with the team regarding this TASK? How are compatibility issues discovered, are each component tested for it..etc And since Zend is an enterprise, I would guess there are formal internal processes involved for even a trivial update on a component? So tell us all about it 🙂
The process is the same one we’ve used successfully for a number of years: we test against each PHP version that Travis-CI supports.
Generally, as betas and/or RCs of a new version of PHP become available on Travis-CI (generally through nightly builds), we start adding them to our Travis-CI configuration in order to identify issues, and either update our code to make it compatible, or report issues to the PHP project when we identify BC breaks.
That process has evolved a ton for us in the last few years, due to changes in PHP 7.1/7.2 and PHPUnit. We have to have mechanisms in place to vary dependencies installed based on the PHP environment being tested.
We’ve come up with a predictable and reliable strategy, but it has only stabilized in the last six months.
First, we update our PHPUnit dependency to allow any of the 5.7, 6.0, and 7.0 versions:
"phpunit/phpunit": "^5.7 || ^6.0 || ^7.0".
In some components, where we haven’t had many changes, we were still supporting PHPunit 4.8, which has meant further updates (more on that later).
– php: 5.6
– php: 5.6
– php: 5.6
– php: 7
– php: 7
– php: 7
– php: 7.1
– php: 7.1
– php: 7.1
– php: 7.2
– php: 7.2
– php: 7.2
– if [[ $TEST_COVERAGE != ‘true’ ]]; then phpenv config-rm xdebug.ini || return 0 ; fi
– travis_retry composer install $COMPOSER_ARGS –ignore-platform-reqs
– if [[ $LEGACY_DEPS != ” ]]; then travis_retry composer update $COMPOSER_ARGS –with-dependencies $LEGACY_DEPS ; fi
– if [[ $DEPS == ‘latest’ ]]; then travis_retry composer update $COMPOSER_ARGS ; fi
– if [[ $DEPS == ‘lowest’ ]]; then travis_retry composer update –prefer-lowest –prefer-stable $COMPOSER_ARGS ; fi
– if [[ $TEST_COVERAGE == ‘true’ ]]; then travis_retry composer require –dev $COMPOSER_ARGS $COVERAGE_DEPS ; fi
– stty cols 120 && composer show
– if [[ $TEST_COVERAGE == ‘true’ ]]; then composer test-coverage ; else composer test ; fi
– if [[ $CS_CHECK == ‘true’ ]]; then composer cs-check ; fi
– if [[ $TEST_COVERAGE == ‘true’ ]]; then travis_retry php vendor/bin/php-coveralls -v ; fi
Let me decipher this for you.
We define some basic details that are not environment dependent: initial composer arguments, and the dependency we will use for pushing coverage reports to the CI job that analyzes those.
Next, we define our jobs. Each job is a combination of a PHP version, and what set of dependencies (DEPS) to use, which is one of either “lowest”, “locked”, or “latest”.
Some jobs define additional environment information, such as whether to run test coverage or CS checks, or additional “legacy” dependencies.
Before we install anything, we determine if we are running test coverage; if not, we disable XDebug, which helps boost both Composer performance as well as testing. We’d prefer jobs execute quickly!
The installation phase is where it all comes together. We first install dependencies from the lock file, ignoring any platform requirements.
This allows developers to update the lock file without worrying about whether a given dependency works on all PHP versions (some packages will remove support for a major version of PHP within a minor release!).
Next, we check to see if a LEGACY_DEPS environment variable was defined; if so, we run a composer update referencing the packages it lists — this is what allows us to install an earlier version of PHPUnit (and, occasionally, other packages) on PHP 5.6 and 7.0!
Next, if DEPS is latest, we run a
composer update, and, if DEPS is lowest,
composer update --prefer-lowest --prefer-stable to test against the latest and earliest versions of dependencies allowed, respectively.
Finally, we list the final set of packages installed for the job.
The script includes
stty 120 to set the output width; if this is not set, composer does not detect a TTY is in use, and spits out only the package names, but not the versions; the latter is very important when analyzing jobs, as it allows us to identify a potential range of dependency versions that led to test failures. (As an example, just yesterday, I discovered jobs that always passed on “lowest”, but not on “locked” or “latest”, and only with PHP 7.1+; by examining the dependencies, I was able to isolate the problem to a change in behavior in PHPUnit that I was then able to provide a change in test setup to accommodate.)
As a maintainer, I can switch to a PHP binary mimicing the setup of a failed job, install the same dependencies using the same sequence of commands the job ran, and run tests myself in order to drill down into them better (e.g., with a debugger, running individual tests, etc.).
>> Is there a pre-defined roadmap with this TASK and hence a deadline?
We defined a GitHub project for the task, but set no deadline.
The reason we set no deadline is that this was also an opportunity for us to touch basically every single component and review any existing open patches.
This process takes time, and there’s no way to meaningfully estimate it.
Some patches you review take a moment or two to understand and merge. Others take hours.
Some components have only a few open patches, if any; others have a dozen or more.
As such, we left it open-ended, but chose to make this a focus of our attention until it was complete.
>> There are 166 repositories (components), handled by 21 people. Are all these people working officially for Zend or do these include voluntary contributors as well, chosen by Zend?
No; the vast majority of these people (the community review team) are volunteers. These maintainers are most often chosen by me based on past activity, but are in some cases recommended by others on the community review team. They provide code review, merge patches, and create releases.
>> What are the main challenges that came through, either for you or the team, during this TASK?
The biggest issue was PHPUnit compatibility with PHP 7.2, and ensuring we could install and run PHPUnit on each of the target environments.
The example Travis-CI configuration I outlined above was the result of a number of iterations, and we learned from each of them… but it took time. Additionally, there are differences in the various PHPUnit versions, which meant we had to adapt existing tests. Fortunately, the PHPUnit developers tend to think about forwards compatibility, and introduce features from one or two major versions down the line to allow you to begin migrating your test suites. Some examples:
- PHPUnit 5.6 or 5.7 added class aliases for namespaced versions of their classes, allowing developers to reference them by the namespaced versions that were released in PHPUnit 6.
- The PHPUnit 4 series allows creating mock objects via
createMock()or a combination of
getMockBuilder(...)->getMock(), allowing developers to migrate to those constructs from the original
getMock()method (which was deprecated in PHPUnit 5 and removed in PHPUnit 6).
- PHPUnit 5 added the
expectException*()methods, which were removed in PHPUnit 6, allowing developers to migrate to those from the original
While having these is great, we found we needed to adopt all the latest features at once if we also wanted to test against PHP 7.2.
The reason is that PHPUnit versions prior to version 6 did not work with PHP 7.2 due to language features PHPUnit used that were marked deprecated in PHP 7.2 (and thus resulted in triggering PHPUnit’s internal error handler).
Generally, we are able to simply add new job entries for a new PHP version, verify they run and pass, and move on. Adding support for PHP 7.2, however, meant updating all tests and our CI processes.
>> While creating a component, what’s the deciding factor – is it because the objectives of the main project requires it, is it because someone in the team identified that such a component would be useful or is also because another framework is offering a similar one and ZF need as well to have it? (I know I’m too curious at times :D)
>> One of the tickets created for this TASK, had the title “Add PHP 7.2 support, drop HHVM”. Can you tell more about that?
- PHP 5.6 and up introduce features not available in HHVM or not compatible with HHVM.
- Usage rates of HHVM are very, very small.
- PHP 7 is really, really fast, making the primary selling point of HHVM mostly moot.
>> Each time a new (major) version of PHP will be out, the components would necessitate another update regarding compatibility. So is there a specific way that the team codes those component to ease such related TASK? How are things done to address TASK in the best possible manner each time periodically?
The End of monolithic framework at Zend?
Ridiculously and unreasonably excited by the fact that today marks the end of the #zf 2.4 LTS release.
End of the monolithic framework, at long last!
— Matthew Weier O’Phinney (@mwop) March 31, 2018
>> Is ZF really now 100%-ly made up of decoupled independent components?
- Users of PHP 5.3; we dropped support for PHP 5.3 and 5.4 when we released our own version 2.5, but, at the time, we had a lot of users who were still on PHP 5.3. Giving them an LTS gave them breathing room to upgrade.
- Users of the monolithic framework liked having a single tarball or dependency to install, and this gave them opportunity to gradually learn Composer and adapt to workflows that use.
- zend-skeleton-installer, which provides prompts for optional features you may want to install, and which removes itself after initial installation.
- zend-component-installer, which is a Composer plugin that checks to see if a component exposes itself as a ZF module (or Expressive config provider)
- zf-development-mode, which allows you to toggle development mode
- zend-mvc, the MVC layer itself
>> I guess it took a lot & a lot of hardwork, energy and time to reach this milestone. Do you want to briefly share your experience through this?
When I tweeted the end of the monolithic framework, what’s interesting is that we have been operating under that paradigm for two years now.
The tweet simply commemorates the fact that on 31 March 2018, the ZF 2.4 LTS ended. 2.4 was the last version shipping as a monolithic framework, and the end of its LTS means we will no longer be updating it.
This simplifies things for us greatly; the LTS releases were complicated to create, and quite error-prone (due to difficulties testing the monolithic framework, lack of automated processes around updating Zend\Version, etc.).
The end of the LTS means we can focus ONLY on the components going forward. Frankly, I’m quite pleased, as it’s the culmination of several years of effort!
>> Expressive 3 provides a middleware microframework approach.
>> Would you say that a middleware approach is better than going the traditional way (MVC approach for example) that most developers are used to?
>> Is there any disadvantage of that architectural approach?
- If you know the ins and outs of an MVC framework, use it. But start learning middleware, and identifying where middleware may help you better implement functionality in your project.
- If you don’t know an MVC framework well, use middleware. It will grow with your needs.
>>In Expressive 1.X, you used a “double pass middleware” approach, whereas going forward with Expressive 3, this is no longer the case, as it now uses lambda middleware. So there must be something that was noted for that change to occur.
>> Could you share your insight about the why behind that change and why you believe the lambda is a better approach that the double pass?
>> Can the microframework, Expressive, be used to create a complex applications that would normally be done by the full-fledged ZendFramework?
>> It’s been a very long time I have not kept myself updated with ZendFramework. Does the v3 still uses an MVC approach or has it been revamped to provide a middleware architecture approach first?
One new feature of interest, however, is that zend-mvc now ships an optional
MiddlewareListener, allowing you to route to middleware (and middleware pipelines!) within your zend-mvc application.