Andrew Welch · Investigations · #devops #php #craftcms

Published , updated · 5 min read ·


Please consider 🎗 sponsoring me 🎗 to keep writing articles like this.

The Case of the Missing PHP Session

A SaaS web­site cre­at­ed with Craft CMS was log­ging out ses­sions after 30 min­utes; here’s how I solved it

Two years ago or so I cre­at­ed a SaaS web­site using Craft CMS called TastyS​takes​.com. It was my first real Craft CMS project, and it aims to be like CraigsList for tour­na­ment pok­er play­ers who want to sell pieces of their action.

All through front-end entry forms, it allows peo­ple to cre­ate accounts, cre­ate pack­ages, list their events, track their results, buy pieces of some­one, and so on and so forth. It also does account­ing, tax forms, noti­fi­ca­tions, and a bunch of oth­er stuff.

Now with the World Series of Pok­er in full swing, nat­u­ral­ly there’s a surge of peo­ple using the site. The only prob­lem is, I was get­ting reports that peo­ple were being logged out of the site.

It’s not a huge deal that people were being logged out, since their browsers remembered the password & login info, but it was annoying.

So I spent a bunch of time look­ing around for pos­si­ble caus­es. The serv­er runs Ubun­tu 16.04 LTS with PHP 7.1, and is man­aged by Lar­avel Forge. The general.php for Craft CMS file has always had the fol­low­ing in it:

'rememberedUserSessionDuration' => 'P1Y',
        'userSessionDuration' => 'P1Y',

The P1Y set­ting is a PHP DateIn­ter­val string that sim­ply means 1 year. But I was hear­ing reports of peo­ple being logged out after much short­er time peri­ods, even what seemed to be under an hour.

I even set the require­Matchin­gUser­A­gent­ForS­es­sion Craft con­fig set­ting to false so that the same login ses­sion could per­sist across mul­ti­ple devices with­out inci­dent, but still no love.

Time to fig­ure out what the heck is going on here.

TL;DR: By default, PHP sessions on Ubuntu will expire after 24 minutes of session inactivity no matter what you set your client-side settings to.

The jour­ney from here was quite a spelunk­ing expe­di­tion into PHP Ses­sions and Ubuntu.

Link Once more unto the breach, dear friends

Time to roll up our sleeves and fig­ure out what’s going on here, which mean a bit of explor­ing, and a bunch of reading.

The first thing we need to under­stand is how PHP Ses­sions work. To quote the bible on this:

Session support in PHP consists of a way to preserve certain data across subsequent accesses. A visitor accessing your web site is assigned a unique id, the so-called session id. This is either stored in a cookie on the user side or is propagated in the URL.

In the case of Craft, the PHPSESSION cook­ie is named CraftSessionId, but it’s the same thing. Here’s what it’s set to in our browser:

Craft­Ses­sion­Id aka PHPSES­SION cookie

The val­ue of this cook­ie is a hash that in this case is set to i152i9u0migsogjk43uidlb114 which is just a key that is used to look up the ses­sion data serv­er-side. Why is it done like this? Because we need some piece of infor­ma­tion to link the vis­i­tor to a par­tic­u­lar ses­sion, but we want the data to be con­trolled and val­i­dat­ed by the server.

So the hash val­ue of the CraftSessionId aka PHPSESSION cook­ie is just a key to look up the actu­al ses­sion data on the serv­er. There are actu­al­ly a num­ber of ways that PHP Ses­sion data can be stored such as mem­cached and Redis, but the default is file-based stor­age. And indeed, if we look in the default direc­to­ry for ses­sion file stor­age on Ubun­tu of /var/lib/php/sessions we see:

forge@nys-production ~ $ sudo ls -al /var/lib/php/sessions | grep i152i9u0migsogjk43uidlb114
-rw------- 1 forge forge    301 Jun 11 20:26 sess_i152i9u0migsogjk43uidlb114

forge@nys-production ~ $ cat /var/lib/php/sessions/sess_i152i9u0migsogjk43uidlb114
5b5e4bf437d8f3b6817ef14f7ef15267__id|s:1:"1";5b5e4bf437d8f3b6817ef14f7ef15267__name|s:22:"andrew@nystudio107.com";5b5e4bf437d8f3b6817ef14f7ef15267__states|a:0:{}5b5e4bf437d8f3b6817ef14f7ef15267__timeout|i:1528783973;5b5e4bf437d8f3b6817ef14f7ef15267__auth_access|a:1:{i:0;s:21:"uploadToAssetSource:3";}

So as you can prob­a­bly tell, there are two entire­ly sep­a­rate pieces of data here. In our Craft general.php set­tings, we told it how long we want the cook­ie to last via the userSessionDuration set­tings. But the cook­ie is just a way to look up the actu­al ses­sion data on the server.

The cook­ie can last as long as it wants, but if the data on the serv­er side is miss­ing, then we have no ses­sion. So now we get in the joys of PHP ses­sion garbage collection.

Link The Garbage Man Cometh

If each PHP Ses­sion cre­ates a cor­re­spond­ing file on the serv­er, as you can imag­ine, we’ll end up with a whole lot of ses­sion files even­tu­al­ly. In addi­tion, we have a way for the PHPSESSION aka CraftSessionId cook­ie to expire on the fron­tend, and we need some­thing sim­i­lar on the back­end for the ses­sion files to expire them.

The com­put­er sci­ence term for this is garbage col­lec­tion: we let a lot of junk build up, and every now and again we clean it up. The default PHP way to do garbage col­lec­tion is every web request that runs a PHP script caus­es it to eval­u­ate and elim­i­nate some expired sessions.

The php.ini set­ting session.gc_maxlifetime defines how long unused PHP ses­sion files will be kept around before they are con­sid­ered garbage and fair game for a chance that the garbage col­lec­tor will clean it up.

This defaults to 1440 sec­onds or 24 min­utes. For exam­ple: A user logs in, brows­es through your appli­ca­tion or web site, for hours, for days. No prob­lem. As long as the time between his clicks nev­er exceed 1440 sec­onds. It’s a time­out val­ue; ses­sions are kept alive if the user is active.

When­ev­er a requests hap­pens with an active PHP ses­sion, the mtime (mod­i­fi­ca­tion time) of the ses­sion file is updat­ed. This mtime is what the PHP Ses­sion garbage col­lec­tion looks at in deter­min­ing if a ses­sion file should be deleted.

I won’t get too deep into how PHP Ses­sion garbage col­lec­tion works, but if you want to learn more about it, check out this arti­cle.

Suf­fice it to say that doing this type of garbage col­lec­tion every web request isn’t ide­al for per­for­mance. The folks at Ubun­tu real­ized this, so by default PHP Ses­sion garbage col­lec­tion is entire­ly dis­abled on Ubun­tu via the php.ini set­ting session.gc_probability = 0.

So if PHP Session garbage collection is disabled on Ubuntu by default, how do the session files get cleaned up?

So obvi­ous­ly there still has to be some way that PHP Ses­sion files get cleaned up, oth­er­wise we’d end up with tons and tons of ses­sion files, which is not great from a disk space or secu­ri­ty point of view. So Ubun­tu includes this cron task in /etc/cron.d/php:

# /etc/cron.d/php@PHP_VERSION@: crontab fragment for PHP
#  This purges session files in session.save_path older than X,
#  where X is defined in seconds as the largest value of
#  session.gc_maxlifetime from all your SAPI php.ini files
#  or 24 minutes if not defined.  The script triggers only
#  when session.save_handler=files.
#
#  WARNING: The scripts tries hard to honour all relevant
#  session PHP options, but if you do something unusual
#  you have to disable this script and take care of your
#  sessions yourself.

# Look for and purge old sessions every 30 minutes
09,39 *     * * *     root   [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi

So alright, this cron task runs the script /usr/lib/php/sessionclean every 30 min­utes (at :09 and :39 on the hour). Let’s have a look at the sessionclean script:

#!/bin/sh -e
#
# sessionclean - a script to cleanup stale PHP sessions
#
# Copyright 2013-2015 Ondřej Surý <ondrej@sury.org>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy of
# this software and associated documentation files (the "Software"), to deal in
# the Software without restriction, including without limitation the rights to
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
# the Software, and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

SAPIS="apache2:apache2 apache2filter:apache2 cgi:php@VERSION@ fpm:php-fpm@VERSION@ cli:php@VERSION@"

# Iterate through all web SAPIs
(
proc_names=""
for version in $(/usr/sbin/phpquery -V); do
    for sapi in ${SAPIS}; do
	conf_dir=${sapi%%:*}
	proc_name=${sapi##*:}
	if [ -e /etc/php/${version}/${conf_dir}/php.ini ]; then
	    # Get all session variables once so we don't need to start PHP to get each config option
	    session_config=$(PHP_INI_SCAN_DIR=/etc/php/${version}/${conf_dir}/conf.d/ php${version} -c /etc/php/${version}/${conf_dir}/php.ini -d "error_reporting='~E_ALL'" -r 'foreach(ini_get_all("session") as $k => $v) echo "$k=".$v["local_value"]."\n";')
	    save_handler=$(echo "$session_config" | sed -ne 's/^session\.save_handler=\(.*\)$/\1/p')
	    save_path=$(echo "$session_config" | sed -ne 's/^session\.save_path=\(.*;\)\?\(.*\)$/\2/p')
	    gc_maxlifetime=$(($(echo "$session_config" | sed -ne 's/^session\.gc_maxlifetime=\(.*\)$/\1/p')/60))

	    if [ "$save_handler" = "files" -a -d "$save_path" ]; then
		proc_names="$proc_names $(echo "$proc_name" | sed -e "s,@VERSION@,$version,")";
		printf "%s:%s\n" "$save_path" "$gc_maxlifetime"
	    fi
	fi
    done
done
# first find all open session files and touch them (hope it's not massive amount of files)
for pid in $(pidof $proc_names); do
    find "/proc/$pid/fd" -ignore_readdir_race -lname "$save_path/sess_*" -exec touch -c {} \; 2>/dev/null
done ) | \
    sort -rn -t: -k2,2 | \
    sort -u -t: -k 1,1 | \
    while IFS=: read -r save_path gc_maxlifetime; do
	# find all files older then maxlifetime and delete them
	find -O3 "$save_path/" -ignore_readdir_race -depth -mindepth 1 -name 'sess_*' -type f -cmin "+$gc_maxlifetime" -delete
    done

exit 0

Welp, so this explains it, then. The rea­son why login ses­sions are tim­ing out is that despite the fact that we told Craft we want­ed the ses­sion cook­ie to last for 1 year, the ses­sion files are get­ting delet­ed every 30 min­utes, for any ses­sion that has­n’t been active in the last 24 minutes.

If the ses­sion file gets delet­ed, we are left with the ses­sion cook­ie hav­ing the key part of a key/value data pair, but the value is entire­ly gone. No ses­sion file, no login ses­sion, it does­n’t mat­ter how long your cook­ie is set to last.

Then, I instantly got skeptical. How could only I be running into this?

When I dis­cussed this with Brad Bell @ Pix­el & Ton­ic, he let out the sigh of a tired old dog lay­ing down to rest ​“I would have thought we’d have heard about this already.”

Indeed, with the pop­u­lar­i­ty of Ubun­tu as a serv­er plat­form, it just seemed so unlike­ly that I was the only one run­ning into this problem.

As it turns out, I’m not. I found a cou­ple of links dis­cussing this, once I real­ized what was going on. Some­times you have to fig­ure out a prob­lem before you even know what search terms to use, ironically:

So why had­n’t any of the P&T folks heard of this either? I think the answer is like­ly because out of the SaaS web­sites with front end entry form logins, many sysad­mins will already be using memcached or redis as a way to store ses­sion data, and thus are also like­ly to have tweaked the session.gc_maxlifetime set­ting while doing so.

For web­sites where users log in via the AdminCP, there’s already an idle task that ends up log­ging them out if they don’t re-enter their pass­word via getAu­th­Time­out.

It may also be that since peo­ple are only logged out after a peri­od of inac­tiv­i­ty, that it’s not real­ly a hin­der­ance. Most browsers remem­ber pass­words anyway.

But it’s still annoying.

Link Let’s Talk Solutions

Fas­ci­nat­ing as all of this dis­cus­sion may have been, so what about a solu­tion to this? Read on!

So here are some solutions:

  1. Set session.gc_maxlifetime in your php.ini file to a large val­ue, equal to what­ev­er you have your ses­sion cook­ies set to live for
  2. Use memcached as your method for stor­ing PHP Ses­sion data
  3. Use redis as your method for stor­ing PHP Ses­sion data

Solu­tion #1 is pret­ty easy, and since the sessionclean script search­es your php.ini for the session.gc_maxlifetime set­ting, it works pret­ty well. You might con­sid­er solu­tions #2 or #3, how­ev­er, because there are nice per­for­mance and scal­a­bil­i­ty gains by using a sys­tem that works from mem­o­ry rather than a file-based solution.

To use solu­tion #2 and switch over to using memcached make sure it’s installed, then just change the fol­low­ing in your php.ini:

[Session]
; Handler used to store/retrieve data.
; http://www.php.net/manual/en/session.configuration.php#ini.session.save-handler
session.save_handler = memcached
session.save_path = "localhost:11211"

To use solu­tion #3 and switch over to using redis make sure it’s installed, then just change the fol­low­ing in your php.ini:

[Session]
; Handler used to store/retrieve data.
; http://www.php.net/manual/en/session.configuration.php#ini.session.save-handler
session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379"

For Redis, you can just install it via apt-get install php-redis, but there is a bit more set­up involved. Check out the arti­cle How to Set Up a Redis Serv­er as a Ses­sion Han­dler for PHP on Ubun­tu 14.04 for details.

Note that both mem­cached and Redis use the php.ini set­ting session.gc_maxlifetime that sets the dura­tion of the ses­sion stor­age as well, so change that to match your cook­ie ses­sion dura­tion, too.

You might be tempt­ed to go for anoth­er solu­tion, which would be to try chang­ing the over­ride­Ph­pSes­sion­Lo­ca­tion Craft con­fig set­ting. After all, if the PHP ses­sion files are no longer where the sessionclean script looks for them, it can’t delete them, right?

While that’s true, remem­ber that by default Ubun­tu has PHP Ses­sion garbage col­lec­tion dis­abled via session.gc_probability = 0. And while you can set this back to its default val­ue of 1 to enable it, then we’re just back to using PHP’s non-per­for­mant ses­sion garbage collection.

We’d just be turn­ing back on what Ubun­tu went through great lengths to work around. So let’s not do that. Pick one of the solu­tions above, and away you go.

So what did I end up going with?

After ver­i­fy­ing that all 3 solu­tions work, I end­ed up going with Redis, because I like the fact that it’ll per­sist ses­sions even if the serv­er is reboot­ed (unlike memcached) and it also can be used with clus­ter­ing and oth­er fun things, should I have the need in the future.

If you do end up going with Redis, and are using Craft CMS, I’d also sug­gest chang­ing the cacheMethod to redis so that Craft can take advan­tage of your shiny new Redis cache, too.

You can then mon­i­tor it to ensure that it’s work­ing as expected:

forge@nys-production /etc/php/7.1/fpm $ redis-cli
127.0.0.1:6379> monitor
OK
1497242867.796951 [0 127.0.0.1:49314] "SELECT" "2"
1497242867.797040 [2 127.0.0.1:49314] "GET" "8c15d45808beb3f5d18176da93a6e893"
1497242867.797520 [0 127.0.0.1:49316] "GET" "PHPREDIS_SESSION:a8cofj5l4eehijlic8k36hngll"
1497242867.797707 [2 127.0.0.1:49314] "GET" "e35dcdf9911497858591ddf4f42dbe16"
1497242867.811311 [2 127.0.0.1:49314] "GET" "dffe4da0cce4155b95fe8360371eb28c"
1497242867.817681 [2 127.0.0.1:49314] "GET" "c7c6dcbff75cd0ba1b63d4ab82504b1c"
1497242867.818137 [2 127.0.0.1:49314] "GET" "f63179c11ec5fbc4198e4e303c3df7a3"
1497242867.818264 [2 127.0.0.1:49314] "GET" "56a0d8f48f1e022b847aca7f7b00ba2c"
1497242867.818383 [2 127.0.0.1:49314] "GET" "74a6ed009475eab91f10f62667f2ad36"
1497242867.818500 [2 127.0.0.1:49314] "GET" "828963ec26955ff6002363a23c683ddf"
1497242867.818615 [2 127.0.0.1:49314] "GET" "7527292f2adc1f03c8a11398a54ed291"
1497242867.819534 [2 127.0.0.1:49314] "GET" "59e4211c4f1e6732a2f37c31317a6913"
1497242867.819751 [0 127.0.0.1:49316] "SETEX" "PHPREDIS_SESSION:a8cofj5l4eehijlic8k36hngll" "2592000" ""
1497242869.757459 [0 127.0.0.1:49318] "GET" "PHPREDIS_SESSION:p7uq0tl1h5keha1re7tnik9r2b"
1497242869.798858 [0 127.0.0.1:49320] "SELECT" "1"
1497242869.798943 [1 127.0.0.1:49320] "GET" "d6912037abb00f0fa4d1029f3221fbd0"
1497242869.821358 [1 127.0.0.1:49320] "GET" "fed528c008af2a4136c5d1048eef466a"
1497242869.890811 [1 127.0.0.1:49320] "GET" "cabb0d3a30387a5169fb174e1aebe5df"
1497242869.891179 [0 127.0.0.1:49318] "SETEX" "PHPREDIS_SESSION:p7uq0tl1h5keha1re7tnik9r2b" "2592000" ""
1497242881.940748 [0 127.0.0.1:49324] "GET" "PHPREDIS_SESSION:svs3omvkpdpodvj4is9k14tsj9"
1497242881.953207 [0 127.0.0.1:49324] "SETEX" "PHPREDIS_SESSION:svs3omvkpdpodvj4is9k14tsj9" "2592000" ""
1497242882.794289 [0 127.0.0.1:49326] "SELECT" "2"

So that’s about it… your users will thank you for not log­ging them out!