it's been bad for a half-decade... :(

at this rate they'll obsolete themselves even if they fix it.

At one point WCG had over 543,000 Users... Considered "Active" recently the number was less than 57,000. When the one little blip of Stat's were released this last week 16,197 Users received a Credit posting.

Considering that many of their current issues are being blamed on "volume" and related issues. How far we have fallen

Respectfully
Bill F

They seem to be posting to this link https://www.cs.toronto.edu/~juris/jlab/wcg.html

April 17, 2026

More problems - this time issues with the data center.
Website/Forum Outage - possibly due to a service interruption in our cloud environment that hosting is working to fix and prevents us from accessing our running instances for maintenance, and may be responsible for other issues although this is currently unclear. Hosting provided an ETA of 1h at 19:00 UTC today April 17th, 2026, and we will keep volunteers posted as we get information and attempt to come back online.
BOINC Backend Outage Ongoing after brief success window on Wednesday, April 15th, 2026 - we enabled the BOINC stats dump after seeing the new architecture handle load and fixing the 503 upload issue. However, in attempting to fix the 404s on the download path by rebuilding the input files and writing them to the tmpfs cache from which downloads are served, we pushed several nodes into various failure states such as SUnreclaim slab memory exhaustion due to the overhead of writing each file to tmpfs en masse, and ill-advised queries run against postgres before EXPLAIN and without paging results to disk that backed up everything else waiting on postgres and caused soft lockup on the node. In addition to issues with the io_method = 'io_uring' setting in combination with our network-attached volumes for the postgres datadir, a setting we may have to change and evaluate before restarting. The naive "should be backup tonight" note in the forums by the WCG Tech was based on having recovered from one of these soft lockups many times in the past few weeks, and before understanding the reason for the initial crash or causing the later crashes on other nodes while attempting various methods of safely regenerating workunits that were throwing 404s on download after being assigned by the scheduler. We will bring the system online as soon as we are confident the results the scheduler will assign have download URLs with files at those paths, and the cluster is stable again.