Tangled in the ThreadsJon Udell, January 7, 2002
A new DSL circuit ends Jon's sheltered lifeThe Internet is an ideal collaborative environment for bad guys. Fortunately it can work the same way for good guys too.
The Honeynet Project estimates the lifespan of a stock system newly attached to the Net to be about one day. By "lifespan" they mean the time it takes to be found, and then hacked. Recently, while switching DSL providers, I got a chance to do the experiment. On my old DSL setup, things were very quiet. That's because my access device, a Cisco 675 (which combines the functions of DSL modem and router), was configured by the ISP to block all inbound requests, and altering that configuration would have voided my agreement with the provider. Not offering public services from my home network was, I decided, a reasonable tradeoff for peace of mind. Once I installed the 675, my software firewalls -- first BlackIce Defender, then ZoneAlarm -- went completely quiet except for occasional outbound alerts.
In my new setup, the DSL modem is a dedicated Westell Wirespeed. (The service, by the way, is called DSL/V, and shares the same wires used by my voice line; you have to filter the voice line, but amazingly it all works like a charm.) The ISP, in this case, drew the line at the Westell. What happened on the other end was entirely up to me: I could have cabled it straight to a PC or router, with or without firewall/NAT protection. I like my new ISP better than the old one, that's why I'm switching, but the difference between these two cases highlights the uncertain state of broadband security:
I've found that the high-speed internet providers either ignore or actively *discourage* end-user security practices. The discouragement usually comes in the form of trying to blame anyone but themselves for system problems; i.e. "You can't access your e-mail? It must be that Linksys router you have that's blocking it. Call us back when you've disconnected and returned you system to the default state."
As it turned out, although the new ISP does not require any security, the folks there were extremely helpful when I did deploy a Linksys, and did run into problems. We quickly eliminated the Linksys, and after Verizon double-checked the circuit, we concluded the Westell -- which I'd picked up on eBay -- was at fault. The ISP reflashed it for me, and then all the lights went green.
By default the Linksys blocks everything inbound, just like the Cisco 675, but now for the first time I was in a position to selectively allow access. I'd noticed that the Linksys offers a "DMZ" feature, but this turned out to be a bit of a misnomer. All it does, really, is fully expose one of the machines on your private network.
They don't explain the purpose of DMZ well, and it really shouldn't even be implemented on a low end device like this. It usually justs ends up getting used as an easy workaround instead of correctly port mapping just the services and ports that need to be set up.
Instead, I mapped port 80 to one of the machines I work on regularly. I disabled the IIS instance that had been running there, and replaced it with this Python microserver:class MyHTTPRequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler): server_version = "Apache/1.3.12 (Unix) mod_ssl/2.6.6 OpenSSL/0.9.4 PHP/3.0.16" def do_GET(self): """Serve a GET request.""" self.send_head() self.wfile.write('ok\r\n') self.wfile.flush()
It pretends to be Apache, and does nothing but accept and log GET requests. Forty-five minutes later, the first visitor -- a NIMDA attack -- arrived:
220.127.116.11 - - [29/Dec/2001 11:31:19] "GET /scripts/..%c0%af../winnt/system32/cmd.exe?/c+copy+c:\winnt\system32\cmd.exe+c:\inetpub\scripts\shell.exe" 200 -
Never mind that that machine seemed to be running an Apache webserver on Unix. Never mind that it was running at an anonymous, unadvertised IP address. Within an hour, it was receiving hostile probes. It wasn't a surprise. I knew, intellectually, that this would happen. But I nevertheless felt a twinge of the visceral gut reaction that is described so well in Clifford Stoll's The Cuckoo's Egg. Clearly, of course, I have been leading a sheltered life.
Forty-five minutes? That long? You must be living on a slow part of the net :-) Frightening, isn't it?
Protection versus accountability
Watching these attack signatures scroll by, as I worked on other things, it was impossible to avoid a sense of outrage. Like Cliff Stoll, I suddenly wanted more than simply to be protected from attack. I wanted to hunt down the attackers and hold them accountable. In one case, an attack traced back to a machine in lab at a university in California. While the attack was underway, I was able to locate a sysadmin, and point him to the compromised machine, which was a departmental NT server providing web access to a small group of Outlook users. The sysadmin was grateful, and I'm sure he immediately cleaned up that server. Of course the compromised server was itself not the original source of the attack, but only a vector.
The procedure I followed -- tracing the machine, identifying administrative contacts, and notifying them -- gave me a measure of satisfaction, but clearly isn't scalable. Couldn't the procedure be automated, though? Before I got very far down that path, I learned that it already has been:
A friend of mine takes the approach of tracking the source and beating on the ISP *in volume*. Admittedly, this doesn't do much for spoofed source packets, but he's had good success in shutting down novice hackers and zombies rather quickly by deploying detecting "agents" on end-user machines and automating the reponse . His program basically reads the logs of various firewall programs and appliances, and uploads them to his database in realtime, which analyzes for attack signatures (by protocol, port number and frequency). He has clients available for Windows and Linux boxes. It's an interesting experiment, if you're curious it's at http://www.mynetwatchman.com.
The vision statement at mynetwatchman.com is, I think, spot on:
Every time your firewall or intrusion detection system logs an event, don't assume the source is the actual hacker. Think of it as a cry for help from a likely victim whose system has been compromised and is just being controlled by a hacker. It's easy to ignore attacks because they don't present an immediate threat--after all, we have a firewall. However, every compromised system is a real and immediate threat to the underlying Internet infrastructure since these systems could be used to attack others and/or to launch distributed denial-of-service attacks (DDoS), potentially incapacitating large portions of the Internet.
In light of these threats, I strongly believe that ALL attack events should be relentlessly pursued.
To achieve this goal, Baldwin says, three challenges must be overcome:
Avoid false reports
Provide aggregated attack reports to responsible parties
There's a lot to like about this approach! What appeals to me most is the way it leverages the same collaborative techniques exploited by attackers. It recruits an army of defenders to stand against an army of attackers, and enables the defenders to pool their knowledge in the same way that the attackers do.
A Cornucopia of Riches
Now that I've got two working DSL circuits, the question arises: what to do? In principle, I can discontinue the first one and save some money. In practice, I can see that's going to be a very difficult decision to make. DSL service feels to me, and to others with whom I compare notes, like a house of cards. It's a long and tricky process to set it up, and if you sneeze the whole thing can come crashing down in an instant. I could voluntarily surrender the now-redundant original connection, but find myself so far unable to pull the trigger.
The newsgroups' resident networking ace, Randy Switt, points out that Nexland's ISB Pro800 turbo is one way to make effective use of a pair of broadband connections -- either cable plus DSL or, as in my case, dual pipes using the same broadband technology. Under normal conditions, the product load-balances the two circuits, boosting effective throughput. (As Randy points out, this won't speed up an individual HTTP or FTP download. But typical websurfing issues a flock of small requests, and these the Nexland box can distribute across the pair of connections.) Should one circuit go dark, the box provides automatic failover to the healthy circuit.
This is a nifty idea. An alternative I'm considering, since I do like the idea of offering public services from my home net, is to reserve one circuit for private outbound use, and another for public inbound use. If both circuits are joined to the same private LAN, there's be no isolation, and the situation would be much like the Linksys' "DMZ." But the circuits need not necessarily share a private LAN. If isolated on separate LANs, a box on the public inbound network would be logically just like a box I might rent at Rackspace or another hosting provider. In this case, failover to recover from a lost circuit would be manual, not automatic, but that's not hard to do. The isolation of the two local networks would be lost temporarily, but the resulting situation would be no worse than any home network that supports both inbound and outbound services.
I haven't yet decided which way to go. If you've got an opinion or advice, I'd love to hear from you. Drop by the networking newsgroups and let us know.
Another Newsgroup Migration
Since 1995, I've corresponded with many of you in a wide-ranging set of online discussions. It's been a moveable feast, migrating from one server to another, and from one conferencing implementation to another. Recently, we've been using a news server maintained by CMP, but they're dropping support for it sometime in January. Meanwhile, the folks who maintain byte.com have been upgrading their JSP-based web-conferencing system. By the time you read this, I expect to have switched from the news server to a corresponding set of web forums at http://www.byte.com/tangledthreads/.
For NNTP junkies (me included), the bad news is that there won't (at least initially) be support for reading and posting from a newsreader. The good news is that some features that were missing from the most recent incarnations of the newsgroups -- notably fulltext search -- are back. It's not a perfect solution, but the truth is that I've used just about every conferencing system ever invented, and none of them is perfect. In the long run what matters to me is the community and its interaction, not the software that supports the interaction. That said, I'm in touch with the developers of the software, and they've already made a number of tweaks in response to issues that arose while I was testing the system. I'm confident that if we run into glitches, they'll be addressed.
Like everyone else, I'm a creature of habit and grudgingly make changes. But I'm certain that the spirit of community which has sustained these discussions will continue to thrive. See you there!
Jon Udell (http://udell.roninhouse.com/) was BYTE Magazine's executive editor for new media, the architect of the original www.byte.com, and author of BYTE's Web Project column. He is the author of Practical Internet Groupware, from O'Reilly and Associates. Jon now works as an independent Web/Internet consultant. His recent BYTE.com columns are archived at http://www.byte.com/tangled/
This work is licensed under a Creative Commons License.