#1

hey web pros.

today you’ve probably noticed some 500 errors from time to time. i have as well. i’ve restarted the digitalocean server 3-4 times which fixes it, but also i’ve seen the issue fix itself with a bit of time.

i’ve gone through the various relevant-seeming threads over at the discourse forums but haven’t come up with anything. cleaned up the docker install, everything’s absolutely up to date, still seeing 500’s sometimes.

any ideas? remember info@monome.org works if the site is down later.

update 11:15 EST: just rebuilt the app, hoping it stays alive now.

#2

Usually there should be something in the server logs. Either the web server or application logs. Since it’s a Ruby app I’d expect there to be a stack trace or something printed somewhere.

1 Like
#3

not 500 errors per se, but erratic loading of the front page since a few hours :
sometimes i see a blank page, sometimes i can access a thread, but when i return to the front page, the tracking is not reset.
and sometimes the user pictures don’t display.
(hoping it makes sense…)

#4

The trouble with 500 errors is that they can be caused by so many different things. Would be very difficult to diagnose without inspection of logs, metrics, etc.

But it sounds like you’re already doing the normal sanitation things to try. Let’s just keep our fingers crossed for now.

#5

Yesterday the forum was acting crazy for me, lots of 500 errors, icons not showing up etc…

It appears to be working fine now though.

#6

ok-- things are still wobbly. here’s from the nginx log:

2016/10/20 10:35:08 [error] 55#55: *42855 connect() failed (111: Connection refused) while connecting to upstream, client: 69.165.153.72, server: _, request: "POST /message-bus/5e304390e1ab439787be9a3d6185c38f/poll?dlp=t HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/5e304390e1ab439787be9a3d6185c38f/poll?dlp=t", host: "llllllll.co", referrer: "http://llllllll.co/t/experimental-music-notation-resources/149"
2016/10/20 10:35:09 [error] 55#55: *39507 connect() failed (111: Connection refused) while connecting to upstream, client: 5.64.107.192, server: _, request: "POST /message-bus/06f35e49f58941888a58214cbb941cd4/poll?dlp=t HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/06f35e49f58941888a58214cbb941cd4/poll?dlp=t", host: "llllllll.co", referrer: "http://llllllll.co/t/field-recording/4644/3"
2016/10/20 10:35:09 [error] 55#55: *39408 connect() failed (111: Connection refused) while connecting to upstream, client: 73.213.38.234, server: _, request: "POST /message-bus/96090ec8afec4263a266ef9ee0455715/poll?dlp=t HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/96090ec8afec4263a266ef9ee0455715/poll?dlp=t", host: "llllllll.co", referrer: "http://llllllll.co/t/soothing-sounds-disquiet-junto-project-0250/4927/13"
2016/10/20 10:35:09 [error] 55#55: *46371 connect() failed (111: Connection refused) while connecting to upstream, client: 174.101.164.170, server: _, request: "POST /message-bus/ce31f02276a541a6b24425d417c87cc9/poll HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/ce31f02276a541a6b24425d417c87cc9/poll", host: "llllllll.co", referrer: "http://llllllll.co/t/500-errors/5022"
2016/10/20 10:35:10 [error] 55#55: *46259 connect() failed (111: Connection refused) while connecting to upstream, client: 155.210.13.111, server: _, request: "POST /message-bus/b9acc36d0a7c47739aa35f8ce0977891/poll HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/b9acc36d0a7c47739aa35f8ce0977891/poll", host: "llllllll.co", referrer: "http://llllllll.co/"
2016/10/20 10:35:10 [error] 56#56: *46515 connect() failed (111: Connection refused) while connecting to upstream, client: 188.37.6.253, server: _, request: "POST /message-bus/7677c32d60684e6c953646b884f69b52/poll HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/7677c32d60684e6c953646b884f69b52/poll", host: "llllllll.co", referrer: "http://llllllll.co/"
2016/10/20 10:35:11 [error] 55#55: *45805 connect() failed (111: Connection refused) while connecting to upstream, client: 88.98.68.3, server: _, request: "POST /message-bus/7e385e15233840e389e47332fd462594/poll?dlp=t HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/7e385e15233840e389e47332fd462594/poll?dlp=t", host: "llllllll.co", referrer: "http://llllllll.co/t/mannequins-rip-rip/4993/17"
2016/10/20 10:35:12 [error] 56#56: *40334 connect() failed (111: Connection refused) while connecting to upstream, client: 70.184.11.97, server: _, request: "POST /message-bus/bf26582371bd4141bc2cb4d4631fc6d2/poll?dlp=t HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/bf26582371bd4141bc2cb4d4631fc6d2/poll?dlp=t", host: "llllllll.co", referrer: "http://llllllll.co/t/experimental-music-notation-resources/149/114"
2016/10/20 10:35:13 [error] 56#56: *46537 connect() failed (111: Connection refused) while connecting to upstream, client: 83.248.138.110, server: _, request: "GET /t/latest-tracks-videos/99/580 HTTP/1.1", upstream: "http://127.0.0.1:3000/t/latest-tracks-videos/99/580", host: "llllllll.co"
2016/10/20 10:35:14 [error] 55#55: *45811 connect() failed (111: Connection refused) while connecting to upstream, client: 84.245.144.188, server: _, request: "POST /message-bus/c76c471e43be47cdb123f93be738d098/poll?dlp=t HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/c76c471e43be47cdb123f93be738d098/poll?dlp=t", host: "llllllll.co", referrer: "http://llllllll.co/"
2016/10/20 10:35:15 [error] 55#55: *46259 connect() failed (111: Connection refused) while connecting to upstream, client: 155.210.13.111, server: _, request: "GET /t/4880.json?track_visit=true&forceLoad=true&_=1476959648090 HTTP/1.1", upstream: "http://127.0.0.1:3000/t/4880.json?track_visit=true&forceLoad=true&_=1476959648090", host: "llllllll.co", referrer: "http://llllllll.co/t/monolase-app/4880"
2016/10/20 10:35:17 [error] 55#55: *36671 connect() failed (111: Connection refused) while connecting to upstream, client: 86.12.163.50, server: _, request: "POST /message-bus/29d6d9a136474007ac514161c4f46b7b/poll?dlp=t HTTP/1.1", upstream: "http://127.0.0.1:3000/message-bus/29d6d9a136474007ac514161c4f46b7b/poll?dlp=t", host: "llllllll.co", referrer: "http://llllllll.co/t/gas-gear-acquisition-syndrome-discussion/3177/62"
#7

rails errors, getting somewhere

Failed to report error: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error. 3 Job exception: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.

augh.

#8

added a swapfile. let’s see if that helps.

alternative fix is to upgrade the ram which will (again) double the server cost.

reference with the exact issue:

https://meta.discourse.org/t/constant-500-cpu-idle/50017/16

#9

Hope adding the swapfile works. Were you able to check the redis logs in response to the rails error?

#10

We are going to be using this software for a work project, so we’ll have some extra eyes on the back end. Doesn’t help you now, but down the road we should be able to.