PDA

View Full Version : HQ Email Comms : Thurs 13th [S*ds Law?]


Keith Lawrence(BSAC)
13-05-2004, 11:40
Hi All

Just to let you know that we?ve had a BT line fault at HQ this morning, all of our email is down, BT and our provider are on-the-case.

I?m REALLY annoyed about this, it?s the first line fault we?ve had in around three years. It?s happened just a few weeks before our ADSL backup circuit comes online? That?s what?s know as s*ds law, this was always a possibility, I?ve planned for it, it beat me by a couple of weeks :-(

Keith L

Keith Lawrence(BSAC)
14-05-2004, 13:27
[Update]

We're still down, we have no internet or email connectivity at HQ. The fault has been traced to a major foul up by our supplier, they are currently working with BT to try and restore something for us. I'm also in touch with HQ and we're working separately to try and get a back-up service running.

Nothing should be lost, our inbound email should be queuing at MessageLabs and we'll pick it up when we come back online. To our volunteers who use <a href="mailto:your.name@bsac.com">your.name@bsac.com</a> email addresses - you will not be receiving anything at present, it has to go via HQ for delivery to you.

Sorry about that folks, I'll let you know when we're back.

Keith L

Keith Lawrence(BSAC)
15-05-2004, 18:32
Well, it turns out that the combined might of BT and our supplier (MCI) conspired to shaft us and cut off our internet connectivity! In short, our suppliers have totally screwed up. They are finally getting things back together, meanwhile Chris (HQ) with the assistance of our IT support have done an emergency patch around to our new ADSL line to get things moving again, as of about midnight (Fri) email was flowing into HQ again.

We may still be having a few problems, we?ve had to make some emergency changes to our network, but we?re hoping that normal service will be resumed on Monday. Then we?ve got to sort the mess out and reconfigure things again when the leased lines come back?

So ? we hope that things are working again, we?re monitoring it. Sorry for the disruption to HQ and our members, it was out of our hands and not really predictable. The IT Team have been working for some time to improve the HQ electronic communications network, when this work is completed in a few weeks time we should be in the position where this cannot happen again in this manner.

Keith Lawrence
BSAC IT Team Leader

John Williams
15-05-2004, 19:43
As a regualr bsac.com email account user I am sorry to report that I've received no bsac.com mail for a couple of days...and since I generally get lots of spam (and loads of the netsky virus) on this account I'd have expected some correspondence.

Is it me...or is it still knackered Keith?

John

John Williams
15-05-2004, 19:56
As a regualr bsac.com email account user I am sorry to report that I've received no bsac.com mail for a couple of days...and since I generally get lots of spam (and loads of the netsky virus) on this account I'd have expected some correspondence.

Is it me...or is it still knackered Keith?

John

Would you beleive it! I just checked to see it it was still not working before I logged off...and all my bsac.com mails are arriving as I type.

Looks OK to me nw Keith

John

John Williams
15-05-2004, 20:01
:=As a regualr bsac.com email account user I am sorry to report that I've received no bsac.com mail for a couple of days...and since I generally get lots of spam (and loads of the netsky virus) on this account I'd have expected some correspondence.
:=
:=Is it me...or is it still knackered Keith?
:=
:=John

Would you beleive it! I just checked to see it it was still not working before I logged off...and all my bsac.com mails are arriving as I type.

Looks OK to me now Keith

John

Oh dear...they are duplicates of mails I had already received but not yet deleted from the server.

No new mails

Still knackerd then Keith

J

Keith Lawrence(BSAC)
16-05-2004, 21:37
Thanks very much for the report John. I?ve had a look (I?m half way home from the Bishop Aukland AGM) and I can confirm that we have half an email system ? we have inbound but not outbound. Your bsac.com emails go IN to HQ, we then send it on OUT to your designated email account. So at present the mail is only getting as far as HQ.

We?re not quite sure where the blockage is just now but we?re working on it and also a work-around that should bypass it if it?s an external problem, we should be able to get email flowing again tomorrow morning. This particular problem has come about as a result of our emergency reconfiguration of the network to get around the original problem!

Sorry for the problems John and all of our volunteers, they were outside of our control. HQ, our HQ IT Support and myself are doing everything that we can to get something working. The REALLY annoying thing is that if this had happened in a few weeks time then the new systems we?re putting in (but haven?t completed yet) would have coped with the failure automatically. That?s s*ds law as they say, BT and MCI decided to ?get in quick? before we were ready for them to screw up our systems :-(

Keith L

John Williams
16-05-2004, 21:46
No need to apologise for something that is clearly not your fault.

Thanks for the update though.

John

Keith Lawrence(BSAC)
17-05-2004, 09:38
Hi John

It should be working again now, we've changed the outbound email route and as of about 07:45 this morning I'm sending and receiving BSAC emails. Somehow our 'virus warning' emails have come back, we normally have them switched off, we'll try and get rid of them for you.

It should be virtually impossible for you to receive a virus via bsac.com email BTW, it is all filtered by MessageLabs before it even reaches us. Don't forget that all modern viruses and worms forge the 'From' address, even if it says it's from a bsac.com address the chances are that's false and it hasn't been anywhere near our email systems.

We're going to leave our email systems on this current 'emergency' setting for a week or two, I'm not going to risk changing it again. We'll set everything up properly when we'd scheduled to do it in a few weeks time.

Keith L

Vic
17-05-2004, 12:18
&gt; Don't forget that all modern viruses and worms forge the 'From'
&gt; address, even if it says it's from a bsac.com address the
&gt; chances are that's false and it hasn't been anywhere near our
&gt; email systems.

SPF can help there...

Vic.

John Williams
17-05-2004, 22:21
Still nothing to report Keith...however I have sent some mails to myself from work today (and used my direct address instead of my bsac.com address - they havn't arrived either!)

If the viruses I'm getting are not via bsac.com - they seem to be coming from ITS staff addresses. We had an email about it recently from someone.

John

Vic
18-05-2004, 01:48
&gt; If the viruses I'm getting are not via bsac.com - they seem to
&gt; be coming from ITS staff addresses.

This is likely.

Most of these worms trawl through your machine to find any email addresses stored there. These are used both as targets for the next wave of attacks, and also as sender addresses to be forged.

If you're unsure about the provenance of a worm, forward it to me (as an attachment) to virustest AT beer DOT org DOT uk. I'll tell you what I can find out about it (which may be no more than the originating ISP...)

HTH

Vic.

Keith Lawrence(BSAC)
18-05-2004, 21:23
Still nothing to report Keith...however I have sent some mails to myself from work today (and used my direct address instead of my bsac.com address - they havn't arrived either!)

I've checked it again John, it's working. The end-to-end time is well under 60 seconds, it takes about 32 seconds within the MessageLabs and BSAC HQ systems before it gets back to your ISP.

It could be that you're sending a message with the subject "Test", that sometimes gets trashed by the anti-virus filter as a blank "Test" message is also a well known virus.

If the viruses I'm getting are not via bsac.com - they seem to be coming from ITS staff addresses. We had an email about it recently from someone.

Not quite, they are coming from ANYBODY with those addresses in their address book, that could be any BSAC member or ex member who has ever communicated with them. I've seen them from me, the NDO, the Chair... none of our PC's are infected, but we're in thousands of address books!

About the only thing you can say for sure John is that they DON'T come from the address that they say they do.

Cheers

Keith L

John Williams
20-05-2004, 22:58
&gt; If the viruses I'm getting are not via bsac.com - they seem to
&gt; be coming from ITS staff addresses.

This is likely.

Most of these worms trawl through your machine to find any email addresses stored there. These are used both as targets for the next wave of attacks, and also as sender addresses to be forged.

If you're unsure about the provenance of a worm, forward it to me (as an attachment) to virustest AT beer DOT org DOT uk. I'll tell you what I can find out about it (which may be no more than the originating ISP...)

HTH

Vic.

Thanx for the offer...I never actually get them through to my machine. My anti-virus warns me about them and I delete them immediately.

I'm probably paranoid about it but I prefer not to handle this sort of material more than I have to do to delete it.

John

John Williams
20-05-2004, 23:01
Seems OK now Keith.

Thanks for checking it out and for the explanation re viruses.

John