The beer is about to get a lot colder..

Posted on November 29, 2011 by danp

When Tim first returned to the UK from his time at Moodle HQ, the first thing he did¹ was to invite me to come to The Great British Beer Festival with him. Apparently i’d taunted him with reports of Real Ale whilst he was in Perth. Well.. it’s time for Tim to get his own back!

After seven years working at LUNS and nine years living in Lancaster, I’ve decided to have a change of scenery and i’m going to be leaving many great friends and colleagues behind to move to Moodle HQ in Perth, Australia. Needless to say, this was not an easy decision to make. (Although the weather like Lancaster demonstrated today could convince me otherwise!)

My involvement with Moodle started with LUNS and it has been a great journey. We were newcomers to the Moodle community with a small project to move a few keen high-schools onto supported hosting platform in 2005. My first post to moodle.org was asking the theme forums for advice on how to do multiple themes on a single codebase and the first patch I remember getting accepted was to help us solve that problem six months later: MDL-6784.² Today LUNS is one of the largest non-HQ code contributors to recent Moodle releases and have provided hundreds of thousands of people with access to Moodle through our various clients – primary school students³, university lecturers⁴, high school teachers, company employees and charity workers. I’ve thoroughly enjoyed that journey and the great people who have made it what it is.

I’m really looking forward to working for Moodle HQ! Despite being so close to Moodle as a developer, I continue to be amazed and motivated by how many people I can touch by working on Moodle. Friends from my days at school and university use Moodle as part of their job on a daily basis, my youngest sister is currently studying and supported with Moodle (unfortunately only at stage 1 of Martin’s pedagogical stages) and I meet more and more moodlers ever day! I’m hoping that my time working as external developer working on the other side of the world to Perth will be a great new perspective to add to the team.

I’ll be working for LUNS for a while yet, but it feels like a good opporunity to thank everyone i’ve worked with over the years at LUNS, clients and community members who accepted my patches. I’m touched by the kind post about my leaving. I expect to continue working with you if more indirectly at Moodle HQ.

Happy Moodling!

Well, the first thing I knew about, anyway [↩]
Although the git history suggests my first fix was actually a horrible JS change which surprises me more than anyone! [↩]
Making their initial steps to learning how to use a computer [↩]
No comment [↩]

Siri Interface to the Moodle Tracker

Posted on November 20, 2011 by danp

Earlier today I came across an interesting project which provides a ‘proxy interface’ to Siri on the iPhone 4S allowing custom plugins to be created to respond to requests from Siri.

Eager to try this I hacked together a Moodle plugin for the siri proxy which would do lookups to the Moodle tracker. So, welcome Siri – the newest aid to the Moodle 2.2 QA testing effort

(The plugin I wrote is available on github. It is my first ruby script and its not particularly elegant as it was just for fun )

Software Metrics and Moodle

Posted on October 11, 2011 by danp

This weekend I attended the phpnw11 conference in Manchester, a good conference with a lot of interesting talks which i’d highly recommend to any php programmer.

Not entirely accidentally, I went to a number of talks focused on testing and continuous integration and came home with quite a lots of bits and pieces I wanted to play with. Sebastian Marek gave an excellent talk on Software Metrics (slides available) as the first track talk of the day. Obviously I can’t summarise his talk, but he introduced us to the concept of the cyclomatic complexity, NPATH, C.R.A.P. as well as WTFs/min as well as tools which can be used to measure them and other assessments of code quality. Sebastian made the point quite strongly that you can’t just use these metrics on their own, but that they could be good indicators combined with analysis of the code.

This got me wondering if the modern parts of Moodle would be analysed more favourably that older parts of Moodle which developers tend to long to refactor away?

Testing 1.9 vs 2.2

So I conducted a test using php mess detector and its code size rules on the question engine in Moodle 1.9 and compared that with master (which includes the rewritten question engine that Tim spent a good portion of the last year working on¹ ).

~/git/moodle$ git checkout MOODLE_19_STABLE
~/git/moodle$ phpmd question/ html codesize > question19.html
~/git/moodle$ git checkout master
~/git/moodle$ phpmd question/ html codesize > question22.html

Results

Moodle 1.9: 315 Problems Found
Moodle 2.2: 233 Problems Found

Moodle 2.2 has better metrics than 1.9!

Well, not quite – in order to get those results I had to remove all the unit tests, which weren’t present in Moodle 1.9 and also tend to be ‘long, dumb’ methods² which trigger just this sort of metric and made the result larger. I don’t really know the code well enough but there may be other factors such as question/ including more code…

The results matched Sebastians point that these metrics can’t be used on their own, but some of the metrics which can be generated might be very interesting datapoints to look at with time. Sonar was a tool demonstrated which could be used in combination with a continuous integration system and tools which generate these metrics to evaluate and report over time – some of which looks really cool and I hope to play with soon.

and I hope he doesn’t mind me using as an example! [↩]
Incidentally Laura Beth Denker also did a great talk at phpnw11 emphasising how you should write good test code avoiding things like conditionals- slides available [↩]

Multi-tenant Moodle without the 2.2 feature

Posted on October 7, 2011 by danp

At LUNS, we designed, built and supported what I still believe to be one of the largest multi-moodle installations in the world. The installation ran from a single codebase and comes with many tough challenges, scaling supporting and upgrading 1000 Moodle installations and 270,000 users can be quite difficult! (I think I have guestimated doing about 8000 Moodle upgrades in my time so far ).

But the question I most frequently get asked is how we achieved a 1000 Moodle install on a single codebase without modifying core Moodle code. This is the simplest part of the operation really and very straight forward to setup. I’ve promised a few people i’d blog about it so:

Configure your webserver to server all hosts from a single codebase directory
Create a config.php file which includes the individual site config depending on the host which is being served. e.g.
```
<?php

$moodle_host = $_SERVER['HTTP_HOST'];

require_once('/etc/moodles/'.$moodle_host.'_config.php');
```
Populate individual Moodle config files with site specific config as you would any other Moodle site

Thats it!

Moodle HQ are currently working on a new core Multitenancy feature, which might provide a different approach for some institutions (although i’m still not entirely clear on its use case over an approach like this).

Moved to WordPress

Posted on October 5, 2011 by danp

Three and a half years ago I decided to start writing a few blog posts about technical bits and pieces I was undertaking. At that time I had a VPS running a few PostgreSQL databases and very little knowledge of the different blogging systems available. I considered a few different options for publishing the blog including Drupal, WordPress and Serendipity. I ended up going with serendipity primarily because it supported PostgreSQL, was a dedicated blogging engine and came recommended by Penny.

On reflection I don’t really think Serendipity was the right choice for me – it doesn’t seem particularly active compared to the alternatives and thus needs some vigilance to keep an eye out for the latest security fixes. The version I was using seemed particularly plagued with php deprecated warnings after upgrading my server to squeeze.

So having recently blogged a few times recently I became a bit more motivated to do something with the blog hosting.. My requirements were relatively simple:

Permalinks *must* be retained (else i’d just start again..)
I don’t care about comments (there haven’t been enough of value to retain and recently a lot of spam..)
I’d like to reduce the maintenance required (if I can get rid of the virtual server that’d be cheaper too)

Blogger

I use a lot of Google services and a couple of my friends use blogger – it seemed functional enough for my needs. Unfortunately, whilst blogger supports import and a custom domains it did not seem to allow for changes to the format of permalinks so I wouldn’t be able to migrate here.

WordPress.com

WordPress itself seems to have all the elements to allow a migration from serendipity. The wordpress.com hosted solution talks of support ‘domain mapping’ but didn’t seem to mention permalink changes. When I signed up for a free blog I couldn’t see a way to change the URL format despite tantalising articles suggesting it was possible.

I contacted the wordpress.com support team asking if changing the permalink url was a service I could pay for (giving an example of my current blog and the wordpress.com post i’d to follow the same format). Two weeks after my request I got a response telling me the difference between the open source project and the wordpress.com hosted solution and that I was confused. I tried to clarify but the support response pointed me to wordpress.org support. I gave up.

Self Hosted WordPress

Having played with WordPress on wordpress.com I quite liked the look of it – clean and simple plus I know it is incredibly popular and an active project. Its frustrating that it doesn’t support PostgreSQL, but I am running another MySQL-only website¹ so I got over my religious views quite quickly.

I completed the migration from serendipity by:

Using the RSS Importer from the full feed. I wasn’t that bothered by much of the metadata, mainly just the post content and dates and this plugin carried out the job admirably.
Changing the permalink structure to match the existing serendipity structure: ‘/archives/%year%-%monthnum%-%postname%.html’

Adding a few redirect rules to keep existing links I cared about working²:

RewriteRule ^feeds/categories/([0-9]+)\-(.*)\.rss$ archives/category/$2/feed [R=301,L]
RewriteRule ^categories/([0-9]+)\-(.*)$ archives/category/$2 [R=301,L]
RewriteRule ^feeds/atom*$ feed/atom [R=301,L]
RewriteRule ^feeds/index.rss2$ feed/rss2 [R=301,L]
RewriteRule ^feeds/index.rss1$ feed/rdf [R=301,L]
RewriteRule ^feeds/index.rss$ feed/rss [R=301,L]

Transferring the images hosted from the blog and altering the links to their new location
Examined the 404 logs to look for things I missed and care about.

All in all this was a fairly straightforward process made quite simple by the fact I have so few blog posts that I could easily verify the results.

The time consuming part

The default wordpress theme allowed me to upload custom header images to display randomly on the top of this blog, its made super easy by having a web based cropping interface and I thoroughly enjoyed spending 3/4 hours last night selecting images from my travels which are displayed above!³

My own archived website from days of my youth when I didn’t know better [↩]
I did not optimise or consider these for more than about 10 seconds, so i’m sure they could be written better [↩]
HINT HINT, RSS readers [↩]

Why i’m moving back to the iPhone

Posted on October 2, 2011 by danp

Over the past eight months I have been using Google Nexus S with Android as my main mobile device, having previously owned an original iPhone & iPhone 3GS. I purchased a Google Nexus S out of gadget love/ tech interest on how Android compares to iOS and then was forced to ‘switch’ fulltime after my own botched ‘repair job’ on my iPhone 3GS.

There are a number of things which attract me to Android:

Open Source: As an open-source developer and user I am ideologically aligned to an open source solution.
Its Cheaper: In the UK at least there tend to be more competitively priced deals on Android devices.
Open App Platform: Another idealogocial point – an open ecosystem sits better with me (though is distinct from an open source OS)
Google Integration: I am a heavy google user and tight integration is appreciated
Apps have much more flexible plugin points – hooks to the Phone/SMS features for example

But after eight months living with Android I can say:

Despite the rhetoric, Google are behaving pretty poorly with their open source platform. iOS 4.3 almost seems more open than Android Honeycomb.
Android continues to be cheaper, although the difference is reasonably small on similar speced hardware. But at the time I switched to iPhone I made the conscious decision to spend more for the luxury item (I moved from paying £10pm to £35pm…) and it turns out i’m still happy to pay extra for the iPhone
The Android marketplace is quite frankly terrible and demonstrates all of what is wrong with a completely open platform. From a consumer point of view the closed Apple App Store is fabulous. My non-technically inclined friends and family can now install and use apps without worrying about the implications. This is quite liberating – and how computing should be!
The only thing I use which the iPhone doesn’t do as well is Google Talk and searching Gmail.
I don’t actually use any apps which use this functionality, despite the attraction

Android: out of sharespace
The main reason I will be moving back to iOS though is slightly intangible, the extra 10% of polish and attention to detail which iOS has in abundance and android misses and frustrates me.

I have two examples of the ‘lack of polish’ to try and make it slightly more tangible:

Yesterday I took a photo with the phone and wanted to send the photo to a friend using MMS. To do this, I usually use the Android ‘share menu’ and select the messaging app. Unfortunately it turned out that yesterday I had installed one too many apps and the messaging app had disappeared off the screen with no apparent way to scroll to it (see screenshot to the right). This is an example of a great feature which Android has over iOS (the share menu) yet made useless for me in practice and is very representive of the kind of things which frustrate me about android.
In my search for a decent music player for Android I installed multiple apps (Winamp, doubletwist, Android MP3) to test out their capabilities. The nexus s comes with earphone with inline remote which can be used to play/pause/skip music (similar to the iPhone). However when I used the remote to skip a tune I had enough of I ended up starting music on one of the other apps – so i’d have two different pieces of music playing at me at the same time, completely ruining the experience. This problem got so annoying that I gave up and bought an mp3 player. Apples approach to background audio playing in apps was, firstly not allow it until iOS 4, then allow a single app to play background audio at a time, the app playing audio is controlled by the system wide controls and can also be streamed using airplay.

Android is much better than anything that was before the iPhone and a worthy competitor, but it turns out that I prefer the more restricted system with attention to detail over the ‘open enough to cripple itself’ android platform – and I haven’t even been subjected to manufacturer hindered OS Updates.

ps. there is a great blog from an iOS user trying out Android in a similar fashion to me: Dinner with Android

A moodle bug a day keeps the doctor away…

Posted on July 26, 2011 by danp

Hi Moodlers..

Apologies – the last entry in this blog was regarding the Moodle Developers Hackfest in the Czech Republic, nearly two years ago!

Moodle Developers in Snow, December 2009

It was a great occasion, we had some big arguments on git, Moodle 2.0 wasn’t released and we didn’t manage to go skiing!

Since then:

Moodle 2.0 has been released (and 2.1)

git has been introduced into Moodle Development workflow

i’ve been skiing (not sure about the Australians..).

we drank the entire czech mint/lime provision in mojitos

Sadly since then i’ve also spent far less time doing Moodle bug fixing than before – instead i’ve been doing a lot of Moodle advocacy to various organisations.

I’ve been missing the bug fixing and decided to start to scheduling myself time to ensure at least one bug fix a week (at the minimum). Fixing bugs and pleasing Moodlers is addictive – so I hope this spirals into many more Wish me luck with keeping the doctor away!

See you on the tracker!

Czech Moodle Hackfest!

Posted on November 29, 2009 by danp

Its less than a week till 16 of us (moodle developers) meet up in the Czech Republic for the first Moodle Hackfest!

I’m really excited, the opportunity to to hack and plan together is a unique opportunity and something we tend to miss at moots as we chat to fellow Moodlers and find out what our users are interested in, drink Mojitos and don’t get so much coding done!

I had hoped to spend a few weeks bug squashing, but sadly my life got in the way.

In any case i’m very interested to hear what you love and hate about Moodle at the moment so the developer community can come up with ways to resolve your problems. If you hate moodle, please tell me why, how can we improve it?

ps. please keep your fingers crossed, I’ve spent months negotiating a new house which I am hoping to own the day I fly to prague!

Ubiquitous Moodle

Posted on September 6, 2009 by danp

Its not difficult for me to realise that Moodle is a popular global project – I work alongside the evidence every day; reading Moodle.org, fearing the volume of bugs which come into the tracker and in CLEO, we recently surpassed 200,000 Moodle users. Buts its actually the small events in life which help me quantify this.

When i’m fortunate enough to go to Moodlemoots the international community of Moodlers is very evident and the passion is incredible.

Often, i meet teachers at completely non-moodle related social events and discover they use Moodle.

While i’m cycling to work and pass children walking to school, I stop and realise these children are likely to be using Moodle now or some time in their future (such is the dominance of Moodle in our region).

This week I discovered my old Sixth Form College is starting to use Moodle. During my time at the college I got a passion for computer programming and also benefited enormously from my first exposure to online learning. I don’t teach people (at least in the formal sense) and so this exposure certainly helps me understand the Moodle philosophy more than I would’ve been able to without.

One of the greatest things about working on this open source project is that my contributions hopefully go some way to benefiting all these users: The moodlers I meet at moots, the teacher I met one time at a party, the schoolchildren passing me on the way to work and the college which gave me many of the fundamentals which have helped me contribute to the project.

Turning my iPhone into a Moodle Server

Posted on June 27, 2009 by danp

Having ended up with a spare iPhone from a recent upgrade I decided to try jail-breaking the old one and see what software was out there away from the restrictions of the app store. I discovered that lighttpd, php and sqlite were all available from the software repositories for download – these three combined are enough to run a Moodle server. So out the window went cleaning my flat and sensible tasks – I had to make my phone into a Moodle server!

Getting the software for moodle installed and configured was relatively painless, the ‘cydia’ software installer appears to use dpkg under the hood, so I installed openssh server and apt through the gui installer and then sshed onto the phone to do the work with a full size keyboard and the moodle server was up and running quite quickly. (More details for configuration below).

Sqlite is a really interesting technology which seems to be making its way into a lot of software and I was quite interested to see Moodle support for lightweight testing sorts of applications and it has made its way into Moodle thanks to the great work of Andrei Bautu in his GSOC project last year. It only exists in the highly unstable Moodle 2.0 development branch so I needed to install this on my iPhone. Development is moving incredibly fast in the Moodle 2.0 branch so I was not at all suprised to see that the sqlite driver was not working. It took a bit of time to find out what the major issue with the driver as it was a silent error. But I eventually found and fixed the major issue.

Sadly, despite successfully installing and passing most of the database unit tests on my development machine with sqlite, some database queries were continuing to cause the iPhone server issues. I spent some bit of time improving the sqlite driver to show more debugging information and get to the bottom of the issue.

After a lot of debugging and irritation, it seems that the sqlite library version (3.3.7) linked to from php has a bug/incompatibility which means it does not like queries like:


SELECT student.id FROM mdl_user student JOIN (SELECT ra.userid FROM mdl_role_assignments ra) roles ON student.id = roles.userid

It’ll report: SQLSTATE[HY000]: General error: 1 no such column: roles.userid

Where as it will work fine with something like:


SELECT student.id FROM mdl_user student JOIN (SELECT userid FROM mdl_role_assignments ra) roles ON student.id = roles.userid

(That was a simple example to try and find the problem – the SQL we have in moodle is a lot more complex that that).

To confuse matters, the sqlite command line tool I was using to test on the phone itself was a newer version (3.6.12), which works absolutely fine with both queries. The same was true on my development machine, which meant that I could install with sqlite succesfully everywhere but the iPhone itself. I assume the php version has been linked to the iPhone OS version – but I am too lazy to check/do something about it!

While I don’t yet have a working moodle server install running on the phone itself, the exercise in improving the sqlite driver has been really useful. I’ve updated the driver in CVS, on recent sqlite versions it is only currently failing 9 of 1298 tests. (CEIL and SUBSTR being the major issues – but they are only used for stats and the admin healthcheck) so its really looking like a really useful option for those situations where a full-grown database server is overkill.
I’ve put a video of the (disapointing) install on youtube and you can find details of the various bits of configuration below.

Configuration Details

I love apt


apt-get install lighttpd php sqlite3 git

I’ve never configured lighttpd before, but a quick search for configuring with php and I made a very simple config with /etc/lighttpd/lighttpd.conf and /etc/lighttpd/mod_fastcgi.conf


JB-Phone:~ root# cat /etc/lighttpd/lighttpd.conf
include "mod_fastcgi.conf"
server.document-root = "/htdocs/moodle"
server.port = 80
server.tag ="lighttpd"
server.errorlog = "/htdocs/log/error.log"
accesslog.filename = "/htdocs/log/access.log"
server.modules = (
"mod_access",
"mod_accesslog",
"mod_fastcgi",
"mod_rewrite",
"mod_auth",
"mod_fastcgi"

)

index-file.names = ( “index.html”, “index.php” )


# cat /etc/lighttpd/mod_fastcgi.conf
fastcgi.server = ( ".php" =>
( "localhost" =>
(
"bin-path" => "/usr/bin/php-cgi",
"socket" => "/tmp/php.socket"
)
)
)

And started the webserver manually with:


lighttpd -f /etc/lighttpd/lighttpd.conf

I created the /htdocs/ directories – and git cloned moodle into /htdocs/moodle and created the moodle config file as mentioned in the sqlite moodle docs:


$CFG->prefix = 'mdl_'; // prefix to use for all table namesV
$CFG->dbtype = 'sqlite3';
$CFG->dblibrary = 'pdo';
$CFG->dbhost = 'localhost';// leave dbhost to localhost (or blank) to store the database file in Moodle data directory

Oh and I was also naughty and made the ‘zip’ php extension an optional item (as it wasn’t in the packed php version):

diff --git a/admin/environment.xml b/admin/environment.xml
index e15a33b..94ed8f2 100644
--- a/admin/environment.xml
+++ b/admin/environment.xml
@@ -262,7 +262,7 @@
<ON_ERROR message="ctyperequired" />
</FEEDBACK>
</PHP_EXTENSION>
- <PHP_EXTENSION name="zip" level="required">
+ <PHP_EXTENSION name="zip" level="optional">
<FEEDBACK>
<ON_ERROR message="ziprequired" />
</FEEDBACK>

Enjoy!

Dan Poltawski's Blog

Author Archives: danp