Andrew Welch

Andrew Welch · Investigations · #devops #php #craftcms

Making the web better one site at a time, with a focus on performance, usability & SEO

2017.06.11 · 5 min read ·

The Case of the Missing PHP Session

A SaaS website created with Craft CMS was logging out sessions after 30 minutes; here’s how I solved it

Logged Out Php Sessions

Two years ago or so I created a SaaS website using Craft CMS called TastyStakes.com. It was my first real Craft CMS project, and it aims to be like CraigsList for tournament poker players who want to sell pieces of their action.

All through front-end entry forms, it allows people to create accounts, create packages, list their events, track their results, buy pieces of someone, and so on and so forth. It also does accounting, tax forms, notifications, and a bunch of other stuff.

Now with the World Series of Poker in full swing, naturally there’s a surge of people using the site. The only problem is, I was getting reports that people were being logged out of the site.

It’s not a huge deal that people were being logged out, since their browsers remembered the password & login info, but it was annoying.

So I spent a bunch of time looking around for possible causes. The server runs Ubuntu 16.04 LTS with PHP 7.1, and is managed by Laravel Forge. The general.php for Craft CMS file has always had the following in it:

        'rememberedUserSessionDuration' => 'P1Y',
        'userSessionDuration' => 'P1Y',

The P1Y setting is a PHP DateInterval string that simply means 1 year. But I was hearing reports of people being logged out after much shorter time periods, even what seemed to be under an hour.

I even set the requireMatchingUserAgentForSession Craft config setting to false so that the same login session could persist across multiple devices without incident, but still no love.

Time to figure out what the heck is going on here.

TL;DR: By default, PHP sessions on Ubuntu will expire after 24 minutes of session inactivity no matter what you sent your client-side settings to.

The journey from here was quite a spelunking expedition into PHP Sessions and Ubuntu.

Once more unto the breach, dear friends

Time to roll up our sleeves and figure out what’s going on here, which mean a bit of exploring, and a bunch of reading.

Exploring Php Sessions Ubuntu

The first thing we need to understand is how PHP Sessions work. To quote the bible on this:

Session support in PHP consists of a way to preserve certain data across subsequent accesses. A visitor accessing your web site is assigned a unique id, the so-called session id. This is either stored in a cookie on the user side or is propagated in the URL.

In the case of Craft, the PHPSESSION cookie is named CraftSessionId, but it’s the same thing. Here’s what it’s set to in our browser:

Php Session Craft Session Id

CraftSessionId aka PHPSESSION cookie

The value of this cookie is a hash that in this case is set to i152i9u0migsogjk43uidlb114 which is just a key that is used to look up the session data server-side. Why is it done like this? Because we need some piece of information to link the visitor to a particular session, but we want the data to be controlled and validated by the server.

So the hash value of the CraftSessionId aka PHPSESSION cookie is just a key to look up the actual session data on the server. There are actually a number of ways that PHP Session data can be stored such as memcached and Redis, but the default is file-based storage. And indeed, if we look in the default directory for session file storage on Ubuntu of /var/lib/php/sessions we see:

forge@nys-production ~ $ sudo ls -al /var/lib/php/sessions | grep i152i9u0migsogjk43uidlb114
-rw------- 1 forge forge    301 Jun 11 20:26 sess_i152i9u0migsogjk43uidlb114

forge@nys-production ~ $ cat /var/lib/php/sessions/sess_i152i9u0migsogjk43uidlb114
5b5e4bf437d8f3b6817ef14f7ef15267__id|s:1:"1";5b5e4bf437d8f3b6817ef14f7ef15267__name|s:22:"andrew@nystudio107.com";5b5e4bf437d8f3b6817ef14f7ef15267__states|a:0:{}5b5e4bf437d8f3b6817ef14f7ef15267__timeout|i:1528783973;5b5e4bf437d8f3b6817ef14f7ef15267__auth_access|a:1:{i:0;s:21:"uploadToAssetSource:3";}

So as you can probably tell, there are two entirely separate pieces of data here. In our Craft general.php settings, we told it how long we want the cookie to last via the userSessionDuration settings. But the cookie is just a way to look up the actual session data on the server.

The cookie can last as long as it wants, but if the data on the server side is missing, then we have no session. So now we get in the joys of PHP session garbage collection.

The Garbage Man Cometh

If each PHP Session creates a corresponding file on the server, as you can imagine, we’ll end up with a whole lot of session files eventually. In addition, we have a way for the PHPSESSION aka CraftSessionId cookie to expire on the frontend, and we need something similar on the backend for the session files to expire them.

Php Garbage Collection

The computer science term for this is garbage collection: we let a lot of junk build up, and every now and again we clean it up. The default PHP way to do garbage collection is every web request that runs a PHP script causes it to evaluate and eliminate some expired sessions.

The php.ini setting session.gc_maxlifetime defines how long unused PHP session files will be kept around before they are considered garbage and fair game for a chance that the garbage collector will clean it up.

This defaults to 1440 seconds or 24 minutes. For example: A user logs in, browses through your application or web site, for hours, for days. No problem. As long as the time between his clicks never exceed 1440 seconds. It’s a timeout value; sessions are kept alive if the user is active.

Whenever a requests happens with an active PHP session, the mtime (modification time) of the session file is updated. This mtime is what the PHP Session garbage collection looks at in determining if a session file should be deleted.

I won’t get too deep into how PHP Session garbage collection works, but if you want to learn more about it, check out this article.

Suffice it to say that doing this type of garbage collection every web request isn’t ideal for performance. The folks at Ubuntu realized this, so by default PHP Session garbage collection is entirely disabled on Ubuntu via the php.ini setting session.gc_probability = 0.

So if PHP Session garbage collection is disabled on Ubuntu by default, how do the session files get cleaned up?

So obviously there still has to be some way that PHP Session files get cleaned up, otherwise we’d end up with tons and tons of session files, which is not great from a disk space or security point of view. So Ubuntu includes this cron task in /etc/cron.d/php:

# /etc/cron.d/php@PHP_VERSION@: crontab fragment for PHP
#  This purges session files in session.save_path older than X,
#  where X is defined in seconds as the largest value of
#  session.gc_maxlifetime from all your SAPI php.ini files
#  or 24 minutes if not defined.  The script triggers only
#  when session.save_handler=files.
#
#  WARNING: The scripts tries hard to honour all relevant
#  session PHP options, but if you do something unusual
#  you have to disable this script and take care of your
#  sessions yourself.

# Look for and purge old sessions every 30 minutes
09,39 *     * * *     root   [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi

So alright, this cron task runs the script /usr/lib/php/sessionclean every 30 minutes (at :09 and :39 on the hour). Let’s have a look at the sessionclean script:

#!/bin/sh -e
#
# sessionclean - a script to cleanup stale PHP sessions
#
# Copyright 2013-2015 Ondřej Surý <ondrej@sury.org>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy of
# this software and associated documentation files (the "Software"), to deal in
# the Software without restriction, including without limitation the rights to
# use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
# the Software, and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
# FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
# COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
# IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

SAPIS="apache2:apache2 apache2filter:apache2 cgi:php@VERSION@ fpm:php-fpm@VERSION@ cli:php@VERSION@"

# Iterate through all web SAPIs
(
proc_names=""
for version in $(/usr/sbin/phpquery -V); do
    for sapi in ${SAPIS}; do
	conf_dir=${sapi%%:*}
	proc_name=${sapi##*:}
	if [ -e /etc/php/${version}/${conf_dir}/php.ini ]; then
	    # Get all session variables once so we don't need to start PHP to get each config option
	    session_config=$(PHP_INI_SCAN_DIR=/etc/php/${version}/${conf_dir}/conf.d/ php${version} -c /etc/php/${version}/${conf_dir}/php.ini -d "error_reporting='~E_ALL'" -r 'foreach(ini_get_all("session") as $k => $v) echo "$k=".$v["local_value"]."\n";')
	    save_handler=$(echo "$session_config" | sed -ne 's/^session\.save_handler=\(.*\)$/\1/p')
	    save_path=$(echo "$session_config" | sed -ne 's/^session\.save_path=\(.*;\)\?\(.*\)$/\2/p')
	    gc_maxlifetime=$(($(echo "$session_config" | sed -ne 's/^session\.gc_maxlifetime=\(.*\)$/\1/p')/60))

	    if [ "$save_handler" = "files" -a -d "$save_path" ]; then
		proc_names="$proc_names $(echo "$proc_name" | sed -e "s,@VERSION@,$version,")";
		printf "%s:%s\n" "$save_path" "$gc_maxlifetime"
	    fi
	fi
    done
done
# first find all open session files and touch them (hope it's not massive amount of files)
for pid in $(pidof $proc_names); do
    find "/proc/$pid/fd" -ignore_readdir_race -lname "$save_path/sess_*" -exec touch -c {} \; 2>/dev/null
done ) | \
    sort -rn -t: -k2,2 | \
    sort -u -t: -k 1,1 | \
    while IFS=: read -r save_path gc_maxlifetime; do
	# find all files older then maxlifetime and delete them
	find -O3 "$save_path/" -ignore_readdir_race -depth -mindepth 1 -name 'sess_*' -type f -cmin "+$gc_maxlifetime" -delete
    done

exit 0

Welp, so this explains it, then. The reason why login sessions are timing out is that despite the fact that we told Craft we wanted the session cookie to last for 1 year, the session files are getting deleted every 30 minutes, for any session that hasn’t been active in the last 24 minutes.

If the session file gets deleted, we are left with the session cookie having the key part of a key/value data pair, but the value is entirely gone. No session file, no login session, it doesn’t matter how long your cookie is set to last.

Then, I instantly got skeptical. How could only I be running into this?

When I discussed this with Brad Bell @ Pixel & Tonic, he let out the sigh of a tired old dog laying down to rest “I would have thought we’d have heard about this already.”

Indeed, with the popularity of Ubuntu as a server platform, it just seemed so unlikely that I was the only one running into this problem.

As it turns out, I’m not. I found a couple of links discussing this, once I realized what was going on. Sometimes you have to figure out a problem before you even now enough to know what search terms to use, ironically:

So why hadn’t any of the P&T folks heard of this either? I think the answer is likely because out of the SaaS websites with front end entry form logins, many sysadmins will already be using memcached or redis as a way to store session data, and thus are also likely to have tweaked the session.gc_maxlifetime setting while doing so.

For websites where users log in via the AdminCP, there’s already an idle task that ends up logging them out if they don’t re-enter their password via getAuthTimeout.

It may also be that since people are only logged out after a period of inactivity, that it’s not really a hinderance. Most browsers remember passwords anyway.

But it’s still annoying.

Let’s Talk Solutions

Fascinating as all of this discussion may have been, so what about a solution to this? Read on!

Php Session Solutions

So here are some solutions:

  1. Set session.gc_maxlifetime in your php.ini file to a large value, equal to whatever you have your session cookies set to live for
  2. Use memcached as your method for storing PHP Session data
  3. Use redis as your method for storing PHP Session data

Solution #1 is pretty easy, and since the sessionclean script searches your php.ini for the session.gc_maxlifetime setting, it works pretty well. You might consider solutions #2 or #3, however, because there are nice performance and scalability gains by using a system that works from memory rather than a file-based solution.

To use solution #2 and switch over to using memcached make sure it’s installed, then just change the following in your php.ini:

[Session]
; Handler used to store/retrieve data.
; http://www.php.net/manual/en/session.configuration.php#ini.session.save-handler
session.save_handler = memcached
session.save_path = "localhost:11211"

To use solution #3 and switch over to using redis make sure it’s installed, then just change the following in your php.ini:

[Session]
; Handler used to store/retrieve data.
; http://www.php.net/manual/en/session.configuration.php#ini.session.save-handler
session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379"

For Redis, you can just install it via apt-get install php-redis, but there is a bit more setup involved. Check out the article How to Set Up a Redis Server as a Session Handler for PHP on Ubuntu 14.04 for details.

Note that both memcached and Redis use the php.ini setting session.gc_maxlifetime the set the duration of the session storage as well, so change that to match your cookie session duration, too.

You might be tempted to go for another solution, which would be to try changing the overridePhpSessionLocation Craft config setting. After all, if the PHP session files are no longer where the sessionclean script looks for them, it can’t delete them, right?

While that’s true, remember that by default Ubuntu has PHP Session garbage collection disabled via session.gc_probability = 0. And while you can set this back to its default value of 1 to enable it, then we’re just back to using PHP’s non-performant session garbage collection.

We’d just be turning back on what Ubuntu went through great lengths to work around. So let’s not do that. Pick one of the solutions above, and away you go.

So what did I end up going with?

After verifying that all 3 solutions work, I ended up going with Redis, because I like the fact that it’ll persist sessions even if the server is rebooted (unlike memcached) and it also can be used with clustering and other fun things, should I have the need in the future.

If you do end up going with Redis, and are using Craft CMS, I’d also suggest changing the cacheMethod to redis so that Craft can take advantage of your shiny new Redis cache, too.

You can then monitor it to ensure that it’s working as expected:

forge@nys-production /etc/php/7.1/fpm $ redis-cli
127.0.0.1:6379> monitor
OK
1497242867.796951 [0 127.0.0.1:49314] "SELECT" "2"
1497242867.797040 [2 127.0.0.1:49314] "GET" "8c15d45808beb3f5d18176da93a6e893"
1497242867.797520 [0 127.0.0.1:49316] "GET" "PHPREDIS_SESSION:a8cofj5l4eehijlic8k36hngll"
1497242867.797707 [2 127.0.0.1:49314] "GET" "e35dcdf9911497858591ddf4f42dbe16"
1497242867.811311 [2 127.0.0.1:49314] "GET" "dffe4da0cce4155b95fe8360371eb28c"
1497242867.817681 [2 127.0.0.1:49314] "GET" "c7c6dcbff75cd0ba1b63d4ab82504b1c"
1497242867.818137 [2 127.0.0.1:49314] "GET" "f63179c11ec5fbc4198e4e303c3df7a3"
1497242867.818264 [2 127.0.0.1:49314] "GET" "56a0d8f48f1e022b847aca7f7b00ba2c"
1497242867.818383 [2 127.0.0.1:49314] "GET" "74a6ed009475eab91f10f62667f2ad36"
1497242867.818500 [2 127.0.0.1:49314] "GET" "828963ec26955ff6002363a23c683ddf"
1497242867.818615 [2 127.0.0.1:49314] "GET" "7527292f2adc1f03c8a11398a54ed291"
1497242867.819534 [2 127.0.0.1:49314] "GET" "59e4211c4f1e6732a2f37c31317a6913"
1497242867.819751 [0 127.0.0.1:49316] "SETEX" "PHPREDIS_SESSION:a8cofj5l4eehijlic8k36hngll" "2592000" ""
1497242869.757459 [0 127.0.0.1:49318] "GET" "PHPREDIS_SESSION:p7uq0tl1h5keha1re7tnik9r2b"
1497242869.798858 [0 127.0.0.1:49320] "SELECT" "1"
1497242869.798943 [1 127.0.0.1:49320] "GET" "d6912037abb00f0fa4d1029f3221fbd0"
1497242869.821358 [1 127.0.0.1:49320] "GET" "fed528c008af2a4136c5d1048eef466a"
1497242869.890811 [1 127.0.0.1:49320] "GET" "cabb0d3a30387a5169fb174e1aebe5df"
1497242869.891179 [0 127.0.0.1:49318] "SETEX" "PHPREDIS_SESSION:p7uq0tl1h5keha1re7tnik9r2b" "2592000" ""
1497242881.940748 [0 127.0.0.1:49324] "GET" "PHPREDIS_SESSION:svs3omvkpdpodvj4is9k14tsj9"
1497242881.953207 [0 127.0.0.1:49324] "SETEX" "PHPREDIS_SESSION:svs3omvkpdpodvj4is9k14tsj9" "2592000" ""
1497242882.794289 [0 127.0.0.1:49326] "SELECT" "2"

So that’s about it… your users will thank you for not logging them out!

Static Electricity

Insights · 2017.06.16

Static Page Caching with Craft CMS

#devops #performance #craftcms

Logged Out Php Sessions

Investigations · 2017.06.11

The Case of the Missing PHP Session

#devops #php #craftcms

Create Craft Cms 3 Plugin

Insights · 2017.05.24

So You Wanna Make a Craft 3 Plugin?

#homestead #PhpStorm #craft-3

Ab Split Testing Second Image Deux

Insights · 2017.05.10

A/B Split Testing with Nginx & Craft CMS

#devops #frontend #split testing

Homestead Sharing

Insights · 2017.05.06

Mobile Testing & Local Dev Sharing with Homestead

#devops #homestead #sharing

Coffee Beans Static Asset Versioning

Insights · 2017.05.05

Simple Static Asset Versioning in Craft CMS

#frontend #devops #craftcms

Buried In Tag Manager Tags

· 2017.04.19

Tags Gone Wild! Managing Tag Managers

#frontend #devops #performance

Homestead Prairie Schooner

Insights · 2017.03.31

Local Development with Vagrant / Homestead

#devops #frontend #homestead

Tidal Wave Disaster

Insights · 2017.03.20

Mitigating Disaster via Website Backups

#backups #devops #craftcms

${ category } · ${ blog.postDate }

${ blog.title }

#${ tag }