ZwiftPower – The coding behind Zwift Community Racing

Zwift has grown from the brainchild of Jon Mayfield into an arguably recognised cycling powerhouse which, I don’t think it is an understatement to say, has revolutionised indoor cycling. In the early days of Zwift, the priority of the company was on making a great user experience. As cyclists, our priority was going fast, which meant races, which early Zwift users created their code to facilitate. This system is now known as ZwiftPower and underpins the majority community of Zwift race results. So let’s take a look under the hood

ZwiftPower is now a core feature of the Zwift Community, and anyone who races even casually on Zwift is aware of this vital racing results system. It can’t really be underplayed how important this system has been to the community of races and riders over the few two years or so. In fact is it is not an exaggeration to suggest that many of the current race teams of Zwift, such as Team Experimental, or KISS Race Team, would not have formed up as they have done without the data and the connections which have been made using ZPZwiftPower

ZwiftPower is now administered by three lunatics chaps; Glen Knight, Christian Weidmann and James Hodges (Who is a little camera shy!)

To put in perspective of how much work this has entailed, Glen had a quick look at their previous conversations, and there have been approximately 21,000 messages regard Zwift power…

So we understand that ZwiftPower is important now, but where did this site and the code come from, and who is behind it? I think to look at that we need to understand what came first and where the need was.

Pre-Zwift Power – Human Power

In the VERY early days of Zwift races, such as the Zwift Training Races, hosted by Christian Weidmann and (myself for a short period), results were done manually. Either by looking at Strava segments, which provided to be highly unreliable or MANUALLY watching a rider cross the finish line on a video, and the more accurate times taken from that! Here is the type of finish video which would form the backbone of Christian’s results

You can see, at this point, we have 80 riders on THE WHOLE of watopia here – these were massive numbers back in 2015, whereas 80 riders are now considered reasonable attendance for an event or set ride! Also, all riders needed to have their race and category in the name, for Christian to allocate them positions and times. These rides were then posted on FaceBook and a few hours later when the numbers had been ground by hand (To say crunched by hand vastly underplays Christian’s efforts) as you can see below

ZwiftPower

Pre-Zwift Power – Rise of the code

From there a coder by the name of Jonathan Lemon wrote the original Zlogger Utility, which has been the coding foundation which ZP is based. This is as aspect which I have found very interesting in looking at the Zwift community. So many of the surface aspects of the projects and programs run by the Zwift riders themselves are based on layers of other projects which have come before. Many of which rely on this original code by Jonathan Lemon.

ZwiftPowerZlogger replaces the human viewer, and “watches” the Zwift riders on the courses and reports every cyclist which cross specific lines in the road, such as the start/finish lines and KOM lines. In doing this Jonathan off loaded a colossal amount of grunt work required in producing race results to lay the foundation for ZwiftPower. Zlogger automated the process of getting race data, but still required a lot of manual work from the race organisers requiring a different configuration for each different course and race length to run reports.

Pre-Zwift Power – Making connections

Zlogger has now been replaced for many people by ZwiftPower but is still vital work on it’s own, as that code is still used to retrieve the data, which is then pushed to ZwiftPower. It is that interface which is the part of the project focused on by Team Experimental rider, and Zwift Community Elder, Christian Weidmann

ZwiftPower

Originally Christian needed some extensions for the ZTR race series, to facilitate the continued race reports, as the ZTR series began to develop a more formal rule set, and in doing so Christian took over from Jonathan as the most active contributor to the data coming into ZP. Glen Knight had the idea of storing data from multiple watchers in a common database and turned to Christian to write the script that transfers the data from zlogger into the ZwiftPower database infrastructure.

ZwiftPower is now based on the idea observing riders performance from multiple watchers placed around the tracks, rather than having computers monitoring riders crossing road markers. Hence why many of the photos showing the inner workings of the KISS Race series, which rely completely on ZP, have many, many instances of Zwift running at the same time, to facilitate this tracking

ZwiftPower

In addition to recording performance, which is kind of vital to a race series, Christian has built other scripts that retrieve rider profiles and scheduled events directly from Zwift which Zwiftpower is then able to populate a lot of the structure you see on the website

ZwiftPower

Zwift has not released a public API. Thus the biggest challenge in working on Zwift power has been figuring out how to access Zwift’s internal API. Several community members independently discovered how to do this. Christian was able to figure a route to the API by looking at Zwift;s dashboard and see how the login worked and how it retrieved information from Zwift’s servers.

ZwiftPower

ZwiftPower is born

If Zlogger is attributed to Jonathan, access to Zwift data, and it’s integration by Christian, then ZwiftPower itself can be attributed to James Hodges. ZP began as a personal project to track the data and relative power output of other racers. That data being weight, w/kg, np, power zones and 20min/5min/1min best efforts. Initially, it was only intended to cover the 2 or 3 races a week James actually took part in himself; then KISS Race Boss came calling…
Glen Knight pitched James some crazy ideas about race monitoring and the idea of building the huge data farm that we know as ZP – that’s when the real work began!!
ZwiftPower
As the sole website developer for ZwiftPower James Hodges is responsible for keeping the site running and implementing new features. ZwiftPower takes the automation a step further than it’s Zlogger underpinnings by using the Zwift event module configuration to determine course and time, then automatically creating race entries based on this information. It was this tracking riders via LIVE data which proved to be one of the biggest challenge. Mainly as the data is often incomplete and has required weekly updates to the site to keep everything running smoothly. But in the same way, as the live data gave James the biggest challenge, so it has been one of ZP’s biggest successes. Now Zwift events are scanned, created and results collated with no human intervention, in turn providing the race data, both in terms of races, but also personal results for rides we have become accustomed too in the Zwift Community
ZwiftPower

Where does KISS feature in all of this?

ZwiftPower

In the early days of the Zwift Community, things were a little like the Wild West; people were learning, and doing, and building practically at the same time. Regarding race series, the central events were ZTR, ZHR, ODZ, KISS & Asia. Glen Knight approached Jonathan Lemon and Christian Wiedmann about centralising multiple watchers over multiple courses, so the data was all consistent but also available all of the time.

With data available all of the time, it was only a small step to start a conversation between James and Glen over the idea of taking this continually available data, and making it “live” data, and thus Zwiftpower LIVE. If the work by Jonathan, Christian and the Zwift pioneers is the bedrock of process, the current ZwiftPower collaboration is the end product, and is now used by most race organisers.

Glen has tracking data on nearly 3000 different events. To put that into context, the 17th of Jan had 33 different races listed as going through ZP. ZwiftPower

But what does that actually take? – Well, Glen hosts 20 physical PCs running various instances of Zwift, Zlogger and their own scripts for data transfer/conversion. On top of that are 5 servers which serve ZwiftOnBoard a feature released at the end of 2016 for Nathan’s live stream allowing you to view riders as they compete in races on Zwift

From a technical side this involves a MySQL server, on the local network so that the internet connection does not effect the transfer of data to the database from the 40+ instances of Zwift. This then replicates to 2 cloud-hosted MySQL servers. One of these servers is used to run scripts for event and rider profile information. The other is used for direct access by the ZwiftPowerfront-end Web server, which is also hosted by Glen – which is should be noted is doing so without charging the Zwift Community anything. This whole enterprise is free for any Zwifter to use!

Things are not finished, every week there seems to be a new challenge for the ZP team. Although in many ways, it seems the biggest challenge that James faces is Glen coming up with another feature request or idea, which I’m told comes genuinely occurs about weekly! However, a very pressing concern for ZP is simpy infrastructure and growth of Zwift. As there have been major changes on Zwift, from when times with 41 riders on Jarvis Island were peak traffic…

ZwiftPowerTo today when it is considered a “quiet time” with 1400+ riders on the island
Zwift power

ZwiftPower currently monitors over 100 data sources across all three courses, and that monitoring is 24/7 – however you chose to look at is, that’s an awful lot of incoming data especially when on peak times Zwift is pushing towards 3000 riders. To put that in context ZP is currently storing records of 41,000,000+ line crossings since March 2016… and before you ask, yes, they have a backup!

When pushed for which part of the project the team are most pleased about there is a bit of a split. Some feel that the technical side, moving from a single server to a data cluster and all of the IT coding is the greatest achievement. Glen, on the other hand, is more impressed with the community result and how ZP has benefitted the race organiser but also the individual user regarding their results.

The team do chat regularly with the Zwift creators Jon Mayfield and Roberto Duarte regarding data and game setup, which is been particularlly imporrtant with live tracking riders on the course, but also gives them ideas about future projects. Given the team are as surprised as anyone at where they were a year ago compared to know, they are very bullish about what they can do in the future!

So what does the future hold for ZwiftPower?

The guys behind ZP still have plenty of ideas they would like to implement given time. These range from tweaks, too much larger changes – sometimes that related to actually implementing a function- ZwiftPower already have the ability to add rider telemetry, live chat data.
Streaming is already happening with ZPOnBoard, but the team want to focus next on stability and integration with ZwiftPower, to allow then additional course information to be displayed.
ZwiftPower
We’ve been working on the way to adding segment PR as a ranking system which can also be used as part of the race setup. We will be adding the ability to support the KISS Zwift Quest event as well as other event types to spice it up a bit
ZP is probably one of the most vital resources to the Zwifting community, and as new system such as the Cycligent Virtual Ranking tool come online, relying on the data from  ZP, that importance is only going to increase!

The future is obviously ultimately affected by Zwift itself. As a result, a major from hope is that Zwift will publish an API that does the equivalent of zlogger without having to run a copy of the full game. This would allow the ZP team to run the infrastructure in the cloud and might go some way to saving Glen Knights electricity bill, but more importantly for the community, would hopefully greatly increase reliability too!

PERSONALLY, I think the throwaway comment about Glen’s power usage is very important. ZP is one of the cornerstones of the Zwift Community. It is easy to say someone will fill the gap if it closed, but that would still be a hugely difficult task. ZP has literally hundreds of man hours in it, a significant investment of time, but also funds for the infrastructure behind it. This is an aspect that is frequently not talked about, upkeep. ZP requires actual hard currency to run. I for one was surprised at the cost, which is being shouldered here by the team, for a community of people they will in reality never meet face to face. Let’s hope that the team keep up the great work, and continue to power the Zwift community for years to come!
From one rider – Thanks, guys!

About James Gill

A runner first, a cyclist second, and when pushed a swimmer. Working as a GP registrar, I found that I went from walking 12miles a day on the hospital corridors to sitting in a chair. Whilst at medical school, I'd run 10km in an evening just for fun, but having lost the general activity required to be a hospital SHO, consciously being active has now become more important - well until I broke myself!

  • Pingback: SIMAD UNIVERSITY()

  • jameszwift

    Thank you so much. Zwift change my life habits and made me healthier, but ZwiftPower made me want to race and really feeling stronger. I would donate 1€ month and i think i wont be the only one.

    THANK YOU ZwiftPower!!!!

  • This is pretty awesome, I’m a webdev/techie and had no idea this amount of work, processing, time and resources are behind Zwiftpower

  • Would be great if maybe they ended up “partnering” with Zwift and got paid – at least a bit – by Zwift to carry on doing the amazing job they currently do. Zwift Power has become such an integral part of the Zwift racing scene that it really does deserve credit and funding!

  • Patrick Foley

    Has anyone done a ZwiftPower users manual? There are many things which are not intuitively obvious (e.g. random magnifying glass, UPG classifications, etc.).

  • anthonycree

    Perhaps a way to send a few $ via paypal (please, no gofundme with their 8% cut), and a notation on ones profile if they’ve contributed. A gold star of some sort so to speak.

  • ZwiftPower is a community project, by fans for the fans. There is no direct funding coming from Zwift

    These guys were some of the first riders to start formalising racing, and needed tools, as mentioned in the piece, to make racing easier, so they made what they needed

    Genuine pioneer spirit.

    In short, our subscription fees go to Zwift. These guys do it for the “RideOns” and sometimes they get a unreasonable level of flack, for a free community project.

  • S.w

    Are we not paying every month anyway?

  • Dan Urban

    Any mean how to donate for at least art of the electricity?

  • Jimmy Salzburg

    Zwiftpower crowd funding/race licenses/whip round, to cover costs and more. Try it, you might be surprised, plenty would be willing to chip in for an insanely cool thing you’re doing!