Skip to toolbar

Dev Diary – Updated 11/07/2018

Home Forums Technical Support Dev Diary – Updated 11/07/2018

Supported by (Turn Off)

This topic contains 14 replies, has 9 voices, and was last updated by  amelias 5 years, 1 month ago.

Viewing 15 posts - 1 through 15 (of 15 total)
  • Author
    Posts
  • #1219962

    timchubb
    Keymaster
    3166xp

    11-07-2018

    Well we have had a busy week and a bit since the last dev diary update….

    Today we have launched the first (of many) minor point releases, yup the site is officially now 2.0.1!  On the original roadmap this was to contain updates to notifications (emails mainly) and the old forums, unfortunatly due to the performance issues we encountered thats not what the meat of this release is about.

    In 2.0.1 we have the following:

    • Home page layout tweaks – @lloyd has made some really slick slideable content which works a treat on mobile devices, such as the latest project update area on the home page, oh yeah projects, place and games now all have a place on the home page as well 😀
    • Massive optimisation and caching on the core pages of the site, the root cause of the performance issues was some unoptimised database and page logic, this has been rewritten and optimised to only run when necessary.  In order to side step the performance issue we threw hardware at it, but this is a very short term fix for something that can be and has been fixed in code, in terms of effciency increases we have gone from over 300 separate database queries to load the home page to approx 28 for most requests, and i have started to scale down the frontend cluster to double check we can handle the load on our preferred server size.  We have also ensured that the cache is as long lived as possible thus reducing the frequency of full page loads which hit the database harder.  The upshot of this is point 3…
    • Realtime updates to the user content driven parts of the home page!, yup comment counts, new places, project activity, game reviews, game ratings and forum activity all now trigger the home page to update no longer is there a lag between activity and it showing.

    Roadmap

    Fixed

    • Lack of caching

    In Progress

    • Forum notifications, first cut will just be for new topic creation and subscribed threads.
    • Tweaks to project image uploads, hope to resolve the issues people have been having with failed uploads.
    • SEO Improvements
    • Rich card support for linked to BoW content

    Upcoming

    • Forum Archives
    • Better notification management

    We will be monitoring server performance but fingers crossed from now on its back to feature development… 🙂

    01-07-2018

    We have had a bit of a blip in site performance starting approx 3pm GMT 30/06/2018, bizarrely it was effecting only one of the servers in the front end pool, but sods lawing what it is, it was the favoured node of the load balancer, so it was getting the majority of traffic routed to it.  Further analysis showed that there was a problem with worker process recycling, when you request a page behind the scenes various calls are made to databases and other systems, and in a perfect world, the site always tidies up after it self closing connections etc., however thats not the world we live in, so the worker processes should get fixed by getting effectivly “turned off and on again” this wasnt happening, so i have altered the thresholds for this virtual off and on to happen and it seems to have improved the situation.

    Onwards we are monitoring the processes and shall be revisiting these thresholds when i have more traffic pattern data (i.e. what happens when you all flock to the site on a sunday morning for your breakfast xlbs 😉 )

     

    28-06-2018
    Hi Guys

    First of all sorry for the slow performance of the site for the last week or so, unfortunately its all part of the tuning and fettling process post go live of 2.0. For those interested, the root cause was a problem with an internal DNS server incorrectly routing DB requests to a public interface rather than one on the virtual LAN that connects the cluster together. But it made us focus on some performance tweaks and changes to our cache mechanism and how we implement certain bits of functionality, which now only run when there is a change, rather than for every page request even if the functionality is not used on the page.

    Now for the fun part the list of what’s been fixed, what’s in progress, and what’s upcoming:
    Fixed

    • Session state is shared between front end servers, which should mean that people finding them-selves logged out should stop happening, and the remember me checkbox on the login page should work as advertised.
    • Lots of DB optimisation, successfully shrunk the daily slow query log from 150mb a day, to 500k, which is a not too shabby 99.6% decrease in slow queries
    • Site performance, as mentioned a DNS SNAFU caused the massive slow down on page requests, that’s now resolved, pages should load between 1~10 secs now on the front end of the site, even quicker if you are signed out and hitting the page cache.
    • Mail feed generator has been upgraded to work with 2.0 data structures, so expect the newsletters to resume in earnest once @warzan irons the last of the style kinks out.
    • Image resizer library has been fettled for speed and performance.

    In Progress

    • Forum notifications, first cut will just be for new topic creation and subscribed threads.
    • Tweaks to project image uploads, hope to resolve the issues people have been having with failed uploads.
    • SEO Improvements
    • Rich card support for linked to BoW content

    Upcoming

    • Forum Archives
    • Better notification management

    Keep on finding the bugs guys, the list is getting shorter but im sure there are some more lurking 😀

    • This topic was modified 5 years, 8 months ago by  timchubb. Reason: 01-07-2018 Performance tuning update
    • This topic was modified 5 years, 8 months ago by  timchubb.
    #1220332

    davehawes
    7768xp
    Cult of Games Member

    Thanks for all the efforts and best of luck with ongoing bug squashing.

    #1220373

    a27cromwell
    Participant
    3127xp

    No idea what you said , except that your fixing things , and for that I thank you and the rest of the crew at head office

    #1220504

    warzan
    Keymaster
    30469xp

    @tgu3 also fixed a bug we noticed in achievements related to projects. It was giving achievements correctly for folk giving them but not for the folk receiving them.

     

    It’s fixed now but obviously there are members who have a lot of nominations who haven’t unlocked achievements yet as the count is only starting from now for everyone.

    I have a small dilemma around this in that we could explore retrospectively adding this but I suspect a week of dev time could easily be sunk into that and I’m reluctant to slow down the overall roadmap of fixes and improvements we are trying to burn through.

    Does anyone have any strong feelings on this either way?

    #1220585

    timchubb
    Keymaster
    3166xp

    @warzan shall see how possible a dB query will be…

    #1220617

    jamjarst
    4011xp
    Cult of Games Member

    Great work @timchubb and everyone else involved. Just in case you don’t hear enough positive feedback but I wanted to add my praise for the new site. I much prefer it to the old one which I was not a great fan of and the Project section is absolutely brilliant?

    #1220898

    soapdodger
    17313xp
    Cult of Games Member

    @warzan I don’t have strong feeling about it people who use it will get the achievements eventually. Though I only noticed this when I logged in and got some achievements.

    I put both of my projects on hold so I can start to think about the hobby challenge,it bumped them to back to the top of the project list, same when I corrected spelling last week.

    This is quite odd, I would have thought it would have only classed an update as a new post. Now I feel a bit bad as people may click for no new content.

    Only realised this due to the achievements.

    #1220948

    warzan
    Keymaster
    30469xp

    Good spot @soapdodger we may have to make the code a little smarter in that area

     

    Thanks for the valuable feedback mate

    #1229445

    timchubb
    Keymaster
    3166xp

    bump

    #1229469

    warzan
    Keymaster
    30469xp

    Really really pleased with today’s  (pretty massive) update @timchubb the homepage seems to be operating beautifully so far 🙂

    Nice work @tgu3 @lloyd 🙂

    #1273632

    Oooo… shiny tech babble. I might not be cleared for this Ultraviolet level stuff. Being a simple Red sector fellow (possibly cleared Orange the longer I stay on the boards) I am confused by the Computer. I know it is my friend and I duly serve it.
    {Paranoia refs for those thinking I’m more than a little off here}

    Good stuff I’ll say. The performance has picked up for the mobile use when I check the site while at work.

    #1303305

    wahidovic
    Participant
    10xp

    no thing is better than when we thanks the other

    thank you

    • This reply was modified 5 years, 4 months ago by  wahidovic.
    • This reply was modified 5 years, 4 months ago by  wahidovic.
    #1310052

    A little odd but after seeing the server was busy I resent a topic for the forum and the earlier entry popped in. Just wondering if you guys were still tinkering and threw some things together (or uploaded a video) and wanted to let you know.

    #1314269

    timchubb
    Keymaster
    3166xp

    Always tinkering ?

    #1340226

    amelias
    Participant
    10xp

    thanks for the improvements and good luck !

Viewing 15 posts - 1 through 15 (of 15 total)

You must be logged in to reply to this topic.

Supported by (Turn Off)