Blog

Open-Sourcing Old Projects

I have made the decision to open-source some of my old projects onto Github. These are ones that have not seen any love in a while.

So with no further ado I am excited to announce the open-source debut of 3 such projects:

  • PaprikaLang – A hand-rolled compiler targeting .Net. The Paprika language itself is also an experiment to disguise certain key functional techniques (like reduce/fold and immutability) and present in them more palatable way for the procedurally minded. Paprika has some esoteric syntax that I have not seen all in one language before. Probably at some stage I will write a bigger post about this; (programming) language design has always been an interesting topic for me.
  • GamutOfBlob – My entry for the Gamedev.net Week of Awesome IV competition which won 1st prize. Coded in only a week, so don’t judge the code too harshly 😉
  • GiftStack – Just me messing around with Apache Cordova and OnsenUI to build a simple mobile app. Horribly unfinished but it does demonstrate that I can do the Javascripts when I have to!

 

So why am I doing this? Really just to show off some of things I’ve done over the years. Why do any of us upload anything to the internet afterall!?

More projects will follow in due course as and when I find the time to excavate them from the darkly depths of old harddrive backups. But I’m pleased that the ball is already rolling!

Week of Awesome IV – Gamut of Blob – Postmortem

Embarrassingly I haven’t used this blog in 2 years and that was for my WoA2 postmortem!

Well WoA4 is over and I am super pleased with my entry. It is somehow more complete and polished than I had anticipated to achieve considering I had a full week of work and a couple of weekend commitments that would see me Macbook-bound for the majority of the weekend.

This year announced 4 themes and contestants were expected to select and incorporate 2 of these for their game: Shadows, Evolution, Ruins, Undead.

A restriction was that Shadows and Evolution had to manifest as gameplay mechanics, whilst Ruins and Undead would need to manifest as graphical theming. Since I am no artist, it takes me a long time to produce art and I have no experience producing art of any kind on a Mac, I decided to opt for the 2 gameplay themes: Shadows & Evolution.

Scope is always an issue in any project. So I took the entire Monday just to let the themes sink in and try to come up with an idea that was appropriate in scope for the free-time I had. I was open to extremely simple games like Flappy Bird but I simply could not marry that up with the requisite themes (evolution, shadows, ruins, undead) in a way that enthused me. Also under consideration was a farming game where veggies are bred and transmuted to achieve a desired phenotype with a supply & demand mechanic driving that, as well as some kind of underwater game like flOw. Both interesting ideas but were too ambitious.

The idea I went with was top-down maze-puzzler with minimalist graphical look featuring cute little blob creatures. Gameplay-wise I was partially inspired by games like Chronotron or the Clank levels from Ratchet & Clank: A Crack in Time. In these games there is a way to generate multiple versions of yourself which must collaborate together to finish the level, things like standing on buttons to open gates. While I did not include their time-travelling mechanic I do have lots of cute little blob creatures running around standing on switches to banish shadows and allow fellow blobs to progress. Blobs can replicate themselves by finding DNA/gene fragments in the level; picking these up will spawn an evolved blob friend, each blob colour has a unique skill which allows it to access parts of the map that your regular unevolved green blob is unable to reach.

day6 (3pm)

The great thing about this idea is that it scales up/down depending on how productive I am: If things go well I can always develop new types of blob which enables new gameplay elements. If things go slowly then I can scale back on my ambitions for lots of different kinds of blob and focus up on just getting *something* done. My original plan included 5 types of blob:

  • Green – The standard unevolved blob
  • Blue – Able to swim and pass through water.
  • Red – Tolerant to extreme temperatures and can walk over lava.
  • Purple – Resistant to the genetic stresses of teleportation and can use teleport pads.
  • Yellow – Speedy, takes long straight paths very quickly and does not stop until it hits a wall.

I probably could have implemented all of these if I had sacrificed some polish but I didn’t want to do that.  In the end I managed all but the Yellow blob, not bad at all! And I don’t feel that the game has suffered at all.

One of the hard-learned lessons from previous years was that starting from a position of no code (just an empty main() method) is tough going. I end up spending a good 2-3 days just knocking out basic infrastructure (resource management, game-states, sprite-rendering, audio, physics, etc). So this year I prepped a very light-weight game-agnostic framework based on LibGDX to handle all that. This was a massive productivity boon and this ‘engine’ wasn’t at all constraining either having received only a minimal amount of frantic hack’n’slash near the end, which I will probably refactor and generalise for the future!

Something that didn’t work out as well I had hoped was the level file format. I used a similar format to my WoA2 game which worked very well for that game. However this time around I had lots of layers to the map and keeping them in-sync was an error-prone head-ache. Some kind of level-editor would have gone a long way to ameliorating this but there wasn’t time for that kind of development. Perhaps for next time I can generalise the concept of a tile-based level editor? Or learn to use an existing one and write an importer for its file format.

Overall, as usual, it was a lot of fun. Very tiring. But a lot of fun.

I have whizzed through playing the other contestants’ games (need to go back and play some of them for longer) and I have to say that I really do not have a good sense for where my game will place amongst them once the judging is over. As usual I am blown away by what people are able to achieve in a week!

Congratulations to everybody who managed to submit an entry. I am looking forward to next year’s competition already! 😉

Save My Toys: A Walkthrough (Week of Awesome II)

First off I am delighted to have placed 5th in the competition overall!

Due to the great response I’ve had to the game, from other competitors and the judges I have decided to record a quick playthrough of my Week of Awesome II competition entry: Save My Toys.

The video quality is bit low but this shows the full extent of the game and the solutions I originally envisaged – although some levels have multiple solutions either by design or not quite intentionally.

Enjoy!

“Week of Awesome II” – Post-Mortem

Amazingly it has been a whole year since Gamedev.net hosted the first Week of Awesome competition. I competed last year and wrote a post-mortem of my game so it’s that time again! A bunch of people have already written up post-mortems, which is great to see. As far as I recall last year only a couple of us did.

This year the rules were much the same: A theme is announced at the start and 7 days with which to implement that into a playable game.

Last year the theme was about what if dinosaurs were still alive. This year the theme was “The toys are alive!“. I’m beginning to notice a sub-theme going on 😉

You can download my game (with a few bugfixes) here: Save My Toys.zip

Save My Toys!

 

What follows is my post-mortem, covering how I approached the problem and how I thought it went after-the-fact…

 

Early Considerations

Before even knowing the theme there are some considerations to make.

  • I am working by myself on this.
  • I am a good programmer (I think) and precious little else – Not an artist, sound designer, level designer… nothing.
  • We have only 7 days available to build/test/polish a game and I have work on 5 of those too.
  • I believe the secret to a good entry is to have a simple idea that is executed well. (The secret to a winning entry might well be a big idea executed well, but that’s something else entirely)
  • When brainstorming ideas it is important to stop and think about how that idea will rack up points from the individual judging criteria. Which include categories like graphics, gameplay, sound, etc.

With these things in mind there is no chance of me making an asset-heavy game. If it requires 3D meshes, skeletal animation, complex sprites, custom sound FX, complex manually-created levels — basically anything like that. It won’t work for me. I am more or less limited to a 2D game with simple, (mostly) static sprites. Last year I got around this by going for a deliberately over-simple MSPaint look and procedurally generated levels, I wanted to try something a bit different this year though.

Additionally it goes (almost) without saying that scope will be a constraining factor here, and feature-creep is an ever-present enemy of any project.

Toy Theme & Brainstorming

The toy-related theme was challenging because it’s not an abstract idea (like “Light vs Dark” say). I concluded that one simply cannot get very far without actually rendering some toys to the screen at some point – and asset production is a major nemesis here.

Being a cutesy theme I wanted some cutesy graphics and was reminded of an artist called Kenny who had posted on GDNet some months prior to tell us all about his free-to-use tilesets that he’d created. I was impressed at the time and quickly hunted them down: http://opengameart.org/users/kenney This guy has done a wonderful thing because are all licensed to Creative Commons Zero which puts them in the Public Domain, as in totally freeballs!

It’s hard to pass up a resource like that, so I very easy settled on the idea of a tile-based game. At this point I brainstormed 3 possible ideas:

  • Sokoban – The idea would be you play as “Mom” or maybe <Insert Generic Child Name Here> tidying away the toys who had up’d and littered themselves around the bedroom during the night.
  • A Rodent’s Revenge variant – Except obviously instead of cat and mouse, the toys would be defending themselves against some kind of enemy (maybe Mom is the enemy?)
  • A Lemmings variant – You control a stream of alive toys who are just trying to get home.

I decided to discard Sokoban. As a core idea it is too simple – I would easily have that game polished in just a few days which would be a waste of the full week. At the same time the prospect of extending the gameplay is daunting. It turns out that the impressive thing about Sokoban is not the gameplay itself, it’s the fact that some poor guy once managed to dream up those devilish maps!

Rodent’s Revenge didn’t really inspire me enough so Lemmings it was! This was always going to be a favourite idea as I have long wanted to make a Lemmings variant.

I began researching for ideas that would be a good fit for a tile-based version. In original Lemmings you assign individual lemmings to specific tasks and they add/remove terrain pixels which alters the flow of those low-res dudes. I had a simpler idea in mind where the Toys are completely self-autonomous and instead you indirectly control them by adding behaviour-altering tiles to the map (like making them jump, turn around, land safely, explode, etc).

I drew my early inspiration from a game called MinimaLemmings; my research indicated that this game has been cloned/re-skinned by others. I didn’t want to clone it but I really liked the core idea of placing jump tiles, so I “borrowed” that and consequently paid homage to them by roughly basing Level 5 “Difficulty Gradient” from their first level. The forces and velocities between my game and theirs aren’t the same, so I was surprised to be able to mimic that level so easily to be honest!

 

What went well?

Sprites & Animations

Kenny’s tiles are great, not only are they a source of inspiration all on their own (mushrooms!), but they work beautifully in the game. At least I think so, I *am* colour-blind so take that sentiment with a pinch of salt 😉

I drew all the toys and their animation frames myself (and the angled mushrooms). Granted they are very simple but I am no artist so I am pleased with the result. I think the cutesy animations work well with the rest of the game.

Level Design

To some extent, this remains to be seen as they haven’t been extensively play-tested yet. For all I know there are flaws in them. That being said, I am pleased with the number and quality of the levels I managed to produce.

The fact that this year I was making a game which would require manually-crafted levels was a major concern to me all the way through, at least until day 6 when I had managed to put a few together finally. In the end I managed to produce 10 levels in the game, in increasing difficulty, although the early levels are really quite easy and exploit low-hanging fruit in terms of level design.

To help ease level production and mitigate the amount of time spent fixing level design flaws close to crunch time I came up with a file format that could convey the shape of the level straight from a text file – quick, easy and low-tech!

For example, here is level4.txt:

<level name="Hazards can be Hazardous!">
    <rescue>10</rescue>
    <map>
        X . . . . X X X X X X X X .
        # . . . X # # # # # # # # X
        # . . F # . S . . . . . . #
        # . . # # # # # # # . . . #
        # . . . . . . . . . . . . #
        # . . . . . . . . . . . . #
        # # # X X # . X . X X # X #
        # # # # # # . # # # # # # #
    </map>
    <spawnerConfigs>
        <spawn amount="10" freq="0.5" />
    </spawnerConfigs>
    <inventory>
        <blockers available="5" />
        <jumpSingle available="5" />
        <jumpDouble available="5" />
        <jumpLeft available="5" />
        <jumpRight available="5" />
    </inventory>
</level>

I opted for “convention over configuration” when it came to levels, meaning I could trivially add new levels just by dropping a file called “levelN.txt” (just replace ‘N’ with a consecutive level number) and the game will load to it when appropriate. No need to register the files with the code.

Time Management

Unlike last year I was more or less on top of my time management this year. My minimum viable entry was essentially ready by some point on day 6 so I had a whole day and half to iron out bugs, design levels and apply other polish.

Part of the reason for this was that I had learnt lessons from last year which minimised the amount of time I spent hunting down terminal bugs. I cpy()’d my Vector2’s, I deferred my Box2d deletes and I had last year’s readme to point me in the direction of good sites to find free fonts and sounds!

Building, Packaging & Deployment

On the week prior to the competition I spent a bit of time nailing the build, package and deployment processes. As this was something that very nearly sunk the ship last year since I left it till the last minute.

First I bundled a complete JRE with my game, so no external dependencies for judges to install! Until the promise of a modularised Java 9 comes out the current JRE is still huuuuge so I had to trim it down a tad. Oracle publish a list of optional files and note one big-ass uncompressed JAR that can be compressed. By eyeing it up, I reckon the JRE is still way fatter than I needed but unfortunately Oracle doesn’t permit you to remove anything further for licensing reasons.

I also needed a way to connect my game with that JRE. A batch script would do it but isn’t very pretty so I found a project called Launch4j which creates an .exe that will launch your JAR for a given JRE – perfect!

Finally, LibGDX (the simplistic game framework I was using) has radically changed their build process since I last played with it. They now use a tool called Gradle, which is a coming together of Groovy, Ant and Maven into a single build tool. It’s quite swanky but I am not familiar with it. So I had some tedious “fun” getting all that to work correctly.

By doing all this the week before, I could just focus on the game itself.

 

What went wrong?

Non-gameplay Screens

Those following my progress updates during the competition would have seen that I posted a few times about getting non-gameplay screens working. My update loop, input handling and rendering were so slapdash that it wasn’t possible for me to handle anything that wasn’t “in-game”. I decided to expend a bunch of time refactoring them to support non-gameplay screens and I ended up re-inventing the old idea of GameStates once again.

Unfortunately I spent so much time doing this that in the end I didn’t have the time to fully exploit the feature. My final game has only 4 states:

  • Start screen
  • Text-based intro screen
  • In-game
  • Text-based end-game screen

Whereas I planned to have highscores and level-selection screens. But hey ho.

The idea and implementation of GameStates is very generic but very useful. I think for next year I will prepare in advance an empty LibGDX project template that includes a GameStates implementation from the beginning.

Box2D is a Physics Engine and will always want to be a Physics Engine!

I coped with this by staying calm. But I know that Box2d is basically a poor choice for the movement controllers in my game.

What I really wanted was:

  • Non-physical horizontal movement but fully-elastic rebound off walls.
  • Gravity (read: physical vertical movement) but completely inelastic landing on floors.
  • Non-physical (but physically “believable”!) jumping movements.
  • Collision detection & resolution (but NOT involving conservation of momentum, etc).

You basically have to fight Box2d to get the non-physical behaviours that you want. On the whole what I want is a colliding kinematic simulation, not a dynamics simulation.

I could probably write one myself, but not in the timeframe of the competition. I decided it would be wiser to fight with Box2d and get the result I wanted.

The magic here basically amounts to tracking velocity yourself outside of Box2d and just keeping the simulation in sync using setLinearVelocity(). Gross but it works.

Additionally, to get completely plastic collision against the ground working well, I had to implement a ContactListener and disregard the contact entirely. To start with I was just detecting the case using the impact normal and calling setLinearVelocity with a zero’d vertical component to remove any bounce, but it was producing visual artifacts (I presume because it was – by necessity – spread across two update cycles with a frame rendered in-between).

Lots of hair-tearing fun to be had all round! Like any framework though, it’s a matter of becoming familiar with its foibles.

My Upload Speed

I always forget how crap my home internet is. It took me 20 – 30 minutes to make an upload. What I should have done is get it all setup on my work PC, remote into it and do the upload from there!

Since I was making incrementally more-finished uploads as the competition drew to a close I ended up losing a couple of hours total just managing my uploads – each one was very tense and I could hardly focus on code while one was going on. I would have liked to churn out a couple more levels, but nevermind.

 

What got Cut?

With more time I would have extended the player’s tool-set to include one-way barriers and explosives (sacrifice a toy to carve a route through for others in its wake – ala traditional Lemmings blowing up).

I would also have liked to explore the lock-and-key mechanic more. I had different coloured lock and key sprites, which would have been used for more complex puzzle logic. I only managed to make a couple of levels involving these, which doesn’t do it justice really.

I also wanted a level where you have a stream of toys that spawn at a slower frequency and you exploit that to split the stream into two teams which head off in different directions simultaneously.

In-game-help – Like last year I wanted to introduce signposts that would popup helpful tips and comedy relief. Some signposts would be off the critical path and only there for the hard-core players to reach.

Gems – Some kind of collectible that would encourage replay and allow for gauging how good a particular solution was.

Water – I had spikes and water was essentially just another hazard. I had pondered the idea that maybe only the ball and the duck could cross water tiles.

One-way Platforms – Currently solid blocks will block from all directions. But there was an occasional need (that I had to work-around in the level design) for a platform tile that could be passed through from the under-side but would be hard like a floor when landed on.

 

Final Thoughts

Once again the competition was a load of fun. It eats a week of your life but the result always feels completely worth it.

I have played many/most/all of the other entries and they are impressive to say the least. Those made with Unity are of particular interest because the pace at which those contestants could churn out gameplay was staggering. I think I will have to spend some time getting to grips with Unity at some point – who knows maybe next year’s entry from me could be Unity powered!

It is interesting actually, the shift in technology choices since last year. This time around there were a lot of people using Unity and other engines. Last year there was a high percentage of Java/LibGDX entries, but only 1 or 2 this year. I expect next year will be even bigger and badder as the competition gains visibility and people improve their skillsets.

Since I was using github to keep my codebase in sync between home PC and travelling laptop my code is (and always was!) available online. But I have spent a bit of time today converting my readme to markdown and cleaning up directories, etc. You can find the repo here: https://github.com/daveagill/WoA2

Now it’s all in the hands of the judges.

Good luck to everybody!

My New DSLR

Canon 700D

A few weeks ago, on a whim, I purchased a Canon 700D DSLR. I have never owned a DSLR before and didn’t have a particularly good grasp of the concepts so I did a bunch of research into the various systems available (from Canon, Nikon, Sony and also the popular Micro four thirds format) before deciding to settle on Canon.

The camera came with a 18-55mm kit lens and I have since purchased a 50mm 1.8f prime lens and a 55-250mm zoom.

The 700D has an APS-C crop sensor, so you’ve got to multiply those focal lengths up by 1.6 to determine the apparent focal length of the final image. Consequently my widest lens at 18mm is effectively 28mm, so I am contemplating also purchasing a wide-angled lens, maybe around 10mm (16mm equivalent).

I took this camera on holiday to Cornwall and got some great shots with it. I’ve also been experimenting with effects like HDR – a technique where you take 3 or more shots at different exposures (but same aperture) and combine them afterwards on a computer to produce an image with a wider dynamic range. I typically use my camera’s Auto-Exposure-Bracketing (AEB) feature with Continuous Shooting enabled to do that quickly (although the AEB has a max range of +-2EV, but that’s not really an issue – if I ever need more I can just compensate the exposure manually).

Here are a couple of hand-held shots I took the other day around where I live with the HDR effect cranked up to extremes to create an arty look:

Water Reel (High Dynamic Range)

Bench (High Dynamic Range)

 

 

 

 

 

 

 

Since buying this camera I have learnt a ton about photography; particularly about how aperture, shutter speed and focal length interact with each other and the final image. Allowing the photographer to control exposure, depth of field, bokeh, starbursts, etc. And about how to apply those to different shots and times of the day.

The 700D is a great device it looks good, feels weighty and solid and has a ton of features, including a touchscreen display (although I turned that off). Before purchasing it I researched it to death and found that it was very well received in all the online reviews. My one criticism of it, after handling and owning it for a few weeks now, is that the plastic shutter release button feels quite cheap and not as smooth to depress as I would like – this seems to be an issue with my specific model since other models in the same family seem to fair much better in that regard, in any case I don’t let it bother me and I might even have a go at spraying it with some lubricant. One thing I am particularly impressed by is the battery life, the thing just goes on forever, which is excellent!

Overall I am very happy with this camera and the vast improvement on my knowledge of photography that I have gained since using it for just a short period of time. With any luck it is only onwards and upwards from here!

Installing CyanogenMod on my Samsung S3-mini

Why did I want to install a new Android?

I’ve been putting up with an annoying issue on my S3-mini (GT-I8190N) for a few months now – the screen will take forever (many seconds) to wake up if the phone’s been sleeping for a while. Even worse is that this will even happen when someone is ringing me!

Most people would probably deal with it, but it bugs the hell out of me and a factory reset didn’t help. So I decided it would be a good opportunity to have a go at installing a different version of Android.

My phone was running with 4.1 ‘Jelly Bean’, whereas the latest version of Android is 4.4 ‘KitKat’, so it was worthwhile an upgrade just to see what’s new in KitKat.

After a recommendation from a coworker and a few minutes of research I quickly settled on CyanogenMod as my Android flavour for no reason other than it seems fairly mainstream.

Gotcha’s!

The first thing I discovered is that it seems that the S3-mini is a relatively obscure device in the world of rooting phones and custom Android versions…

First off, CyanogenMod is not officially supported on the S3-mini (yet it is on the regular S3, the first Galaxy mini and the S4-mini) fortunately NovaFusion has a respectable-looking unofficial port.

Second of all I really struggled to find any a way to root the phone (and it turns out I didn’t even need to!). I tried a number of techniques – all failed.  I didn’t get to try this CF Rootfile approach but that was next on my list. What did work for me was VRoot, a one-click rooter for Windows which worked perfectly after many other one-click rooters had failed; you just have to get over the fact it’s a poorly translated version of a Chinese tool.

Only after having rooted the device did I spot that it wasn’t even necessary! Closer inspection of CyanogenMod’s instructions for the regular S3 revealed that Samsung devices come with a special ‘Download Mode’ (press Power, Home and Volume Down together) from which you can install a custom Recovery image.

‘Download Mode’?

My understanding is that Samsung’s ‘Download Mode’ is akin to Fastboot mode which could only be enabled from an unlocked bootloader, for which you would need a rooted phone. So Samsung kindly side-step all that.

Download Mode allows you to flash ODIN-compatible images (ODIN is a protocol and software package used internally by Samsung and since leaked to the wider world). Rather than using the ODIN software to perform the upload I used Heimdall, as recommended by CyanogenMod’s instructions.

Flashing a custom Recovery image

The image we want to flash isn’t Android itself, it is a custom Recovery image. From which we can perform a backup (before proceeding further) and then do the actual install of a new Android system.

The recovery image is a small image that is ordinarily used to perform factory resets and to handle the OTA updating, as well as a few other things. Booting to a recovery image first is just like when you boot to a CD-ROM, Flash-drive or hidden recovery partition first when you are installing a regular OS on a regular computer.

However, the stock recovery image that comes with most phones does not expose functionality for installing custom Android OS images, or for doing full system backups. This is why we need a custom Recovery image.

The most popular Recovery image seems to be clockworkMod and that is what I used.

For the S3-mini I had to deviate from the flashing instructions given by CyanogenMod. The following command would not work:

sudo heimdall flash --RECOVERY recovery.img --no-reboot

I worked out that the –RECOVERY argument refers to the ID of the partition you want to flash, my recovery partition was just called “19” so I had to use this command:

sudo heimdall flash --19 recovery.img

A brief Googling indicated that this seemed to be true for other S3-mini users too. Nevertheless I would recommend double-checking! To list the partitions I ran this command:

heimdall print-pit --verbose

And then I just looked for one that referred to a recovery.img and took its ID (19).

Making a backup

Before going any further I wanted a backup of the stock Android image, so I could recover to this if CyanogenMod did not work, or if I screwed something up.

First, boot into the recovery image by pressing PowerHome and Volume Up together.

Within the menu there is an option for making a backup. Actually there are two options, one will put the backup on the phone, the other will but it on the external SD-Card. Obviously choose whichever has the most space available. For me that was the external SD-Card, which is probably the ideal place for it anyway.

I went as far as to copy this off the SDCard onto my PC for safe keeping.

Install CyanogenMod already!

Finally, time to install CyanogenMod!

The image .zip must be placed on the root of the SDCard. I used adb to put it there but if you have an external (micro) SD-Card and reader then you can just drag-and-drop.

It is also important to pre-load some Google Apps, otherwise you’ll have Android but no apps (no Play Store app, GMail, Chrome, etc). Luckily NovaFusion have several app-bundle options – I just went for the full zip bundle, nice and easy. Just place this zip alongside the Android image zip on the root of the SDCard.

The rest is easy. Just boot into the recovery image and there is an option to install new zip packages. Install the OS first and then the app-bundle.

Et Voila!

Final thoughts…

The experience was relatively painless and went smoothly – no accidental bricking of my phone!

CyanogenMod works well – I have been using it for about a week now – and there haven’t been any serious issues. One time I had an email syncing issue which a restart fixed, it is far from clear whether CyanogenMod actually had anything to do with that though.

Only one or two minor things do I miss from Samsung’s customised Android. All of which I’m sure I could achieve using freely available apps. One of them is having a ‘home’ screen which is not the far-left one, the default android Launcher (Google Now) always chooses the left-most screen to be the target when you press the Home button. So far these don’t bother me 🙂

Best of all? My screen issue is now fixed!

New PC!

So I am the happy owner a shiny new self-built PC and so far I’m very pleased with it.

This is the first time I have built my own computer and, really, this is the first desktop PC I have owned (although I pretty much exclusively used my parent’s desktop when I lived at home). Until now I have been a laptop-man. I’m on my second laptop currently and it’s probably close to being replaced with a shiny new one soon.

Since I’ve been working from home for the past few months I wanted a proper PC workstation. My fiancée had expressed an interest in a new PC also. So I set to work to figure out how to build one. I made the decision to do an own-build because I think it’s an important rite-of-passage for a Computer Scientist that I had not yet experienced and I was minorly teased at University for the fact that I had never done it and exclusively used my good ol’ laptop. So I was determined that was the route for me.

It turns out that dabs.com has everything you could need and at unbeatable prices, there is virtually no point in shopping around because Dabs prices are *that* good!

After a bunch of research and comparing products I eventually settled on the following spec:

Case: Corsair Carbide 300R Gaming Case (ATX)
PSU: Corsair TX650 v2 650Watt
Mobo: Gigabyte Z87-D3HP Intel Z87
CPU: Intel Core i5-4570 3.2GHz Haswell Quad-Core
RAM: Corsair (2x 4GB) Vengeance Pro Blue DDR3 1600MHz CL9
SSD: Sandisk 256GB Ultra Plus SATA 6Gb/s
Optical drive: Samsung 24x SH-224DB DVD-RW
GPU: Gigabyte Radeon 7790HD 1GB
Monitors (both): LG 22EA63V 22″ 1920×1080 IPS LED
OS: Windows 8.1

I decided to go with the 1150 socket for the Haswell chips for future-proofing. I decided I didn’t go to for the high-performance (read: incredibly expensive) socket and so I’ll stick with the more mainstream stuff.

Also, I have a general fondness for NVidia but bang-for-buck goes to AMD and the 7790HD is a good little performer that should be able to handle any current game, not that I’m a massive PC gamer (I prefer my PS3) but it’s nice to have the option available and maybe I’ll play with some advanced graphics stuff at some point, who knows.

The build itself went without any troubles, no wiring issues, no coloured smoke, no snapped mobo/case, no compatibility issues. Everything worked first time.

I’m still on the hunt for some good but inexpensive speakers, probably 2.1 with the subwoofer. Currently I’m using some really old speakers that look very dated but still sound great. I also want a new keyboard, I’m thinking about getting a mechanical one but I haven’t found one that I really like yet although the Razer Blackwidow Ultimate might be a good contender.

On Becoming Engaged!

I am now engaged to my girlfriend-come-fiancée of 5 years!!

We’re very pleased!

I proposed on Thursday October 24th at Anglesey Abbey in their Dahlia garden and successfully managed to conceal the ring in its box for the whole trip until that moment – so it was total surprise to her! 😀

Everybody always asks if I got down on one knee – and I did! Since it makes for a bit of a spectacle I waited until everybody else had gone away first so we could have our moment in private.

I bought the ring online. It’s platinum with a Morganite central stone surrounded by small diamonds and sets of diamonds embedded in the band also. It’s lovely to look at:

Engagement Ring

First-time fun with an Arduino

So the other day I got an early birthday present courtesy of my lovely girlfriend, an Arduino Mega2560 purchased from CPC.

I’ve only had a few hours to play around with it so far, but I have a basic circuit and Sketch (which is Arduino parlance for the C program that is to be uploaded to the Atmega chip), as you can see…

First Arduino Project

It is really basic at the moment, the program listens for input from the button and every time you press it either pauses or unpauses the PWM (Pulse Width Modulation) in-and-out fading of the LED.

The button is wired to a 10k pull-down resistor. The Arduino actually has the ability to do this itself, you get to choose whether you pull-down yourself or leave it up to the Arduino – Actually I wonder if it is the Atmega chip doing the pulldown, rather than the Arduino.

Every time you toggle the pause/unpause it sends a serial communication through the USB back for display on the Serial monitor (provided as part of the Arduino IDE). It’s really easy to do, but quite a powerful feature built-in.

I need to think up some interesting projects to do with it – Maybe some kind of robot? A robot arm?
I have a RaspberryPi, maybe I can achieve something interesting if I wire them together?

The WordPress Scheduler: wp-cron.php

So, I recently noticed that my site was not doing its automatic backups. In fact it had never done a backup and I simply had not bothered to check!

I use UpdraftPlus to do my site backups, it is a WordPress plugin that zips up the database and uploaded files and sends them to you (I have it send them to my DropBox). After a bit of investigation and following some of the debug steps on their site I came to the conclusion that it must be WordPress’s internal scheduler to blame. Without the scheduler the backups will never get kicked off and other things like checking for updates may be failing too. Question was, why was the scheduler borked?

To understand this we first have to understand how the scheduler works in WordPress.

It basically takes the form of wp-cron.php, a little script that checks to see if there is any scheduled work to do and, if there is, then it performs the necessary processing. This script is called whether a page is loaded on a WP site.

Already, this has some consequences (and some potential red-herrings for my investigation). Clearly if the cron script is only invoked when there is site activity then for a lowly unloved site like mine the cron script isn’t going to get called very regularly and scheduled tasks are not going take place on-time. Maybe the sheer amount of spam comments I get would keep things ticking over, I don’t know. One day, when this site goes viral becomes an internet sensation, the vast quantity of page views would actually keep the cron script too busy, clearly it doesn’t need to be called hundreds of time a minute! In fact as I understand it, in previous versions of WP concurrent invocations could cause problems; a problem that is solved these days using locks.

You might wonder whether page loading is slowed down for those people who happen to visit the site when there is a scheduled task to perform. Fortunately this is not the case. The cron script is initiated concurrently.

WordPress launches the concurrent task by making an HTTP request back to itself, thereby spawning a second server-side thread to handle the request. Something like this:

GET: http://www.yoursite.co.uk/wp-cron.php

Unfortunately this could fail for all kinds of reasons. LIke not being able to resolve the DNS name, for example.

I use a shared hosting provider. This website is hosted on some server alongside a bunch of other websites. The overall server configuration is therefore out of my control. It turns out that a lot of hosting providers (including mine) disable loopback requests. In other words the server is unable to invoke a request on itself. Apparently this is for security reasons, perhaps to prevent some kind of infinite request loop. Personally I’m inclined to disagree, I cannot think of any particularly good reason to do this; there is nothing malevolent that the machine could do by calling back to itself that couldn’t also be achieved externally; they’re just web calls after all!

Initially I wasn’t sure if this was really the case for me. I couldn’t see anything in the logs about failed attempts to invoke wp-cron. But I had a script tat makes HTTP GET calls and using that I proved that it was indeed the case and showed empirically that such failures (which were request denied errors) are not logged for some reason.

So what is the solution? Well there are two possible solutions…

One is that WordPress actually has a second way to trigger concurrent work, you just have to enable it. It works by sending the user on a redirect and hijacking their original request to perform the cron work. I don’t imagine that is the most pleasant user experience for the person visiting your site though.

Another solution – the one I opted for – is to invoke wp-cron.php periodically from a genuine cron-job configured on the server. For many people this may not be a viable option if they don’t have access to cron, or if they lack the technical skills to use it.

Luckily my provider exposes a UI through CPanel that lets me configure up to 3 scheduled tasks. Of course I only need one for this and it would be trivial to write some kind of program that used a single cronjob but multiplexed that out to emulate an unlimited number of scheduled tasks.

The ideal way (perhaps) to have cron invoke wp-cron would be to use wget to make a web request:

/usr/bin/wget http://yoursite.co.uk/wp-cron.php

Alas, that would obviously still involve the loopback, from which I am blocked, so it was not an option for me. Instead I just run php directly over the script. Which seems to work just fine:

 /usr/bin/php5 /path/to/wp-cron.php

I configured the cronjob to run every half an hour. A lot of people seem to run it more frequently even than that! Seems like overkill to me. I suspect you could get away with running it less than once a day in reality.

So now I have smooth-running automatic backups of my WordPress database and files 🙂

It only seems appropriate to use the scheduler to publish this post. It was actually written on September 8th 2013, but as I’ve recently published a post just yesterday I will schedule this one for lunchtime tomorrow.