One of the, ahem, pleasures of working at a startup is that you often have to learn something new in a hurry when something goes wrong. This happened to me the other day when people on our office network suddenly stopped being able to get to anywhere else on the internet or our internal network.
After a bit of troubleshooting, I found out that we still had inbound and outbound connectivity, but DNS had intermittently stopped working - so, since normal people don't carry a long list of IP addresses in their heads, their access to anywhere via its host name was gone.
We had all our office network clients pick up their DNS server via DHCP from our LinkSys AV-082 router. I tried switching the DNS servers defined in the router to OpenDNS in case our ISP's DNS servers were having a problem, but the problem persisted.
Some internet digging and talking with a couple of folks who know a lot more about networking than me led me to the conclusion that the router's DNS and DHCP handling was acting up. Since it still seemed to be working fine as a regular router, I decided the best thing would be to turn off the router's DHCP server and run DNS and DHCP from one of the office servers instead. I set up the people in the office to point directly to OpenDNS to get them going while I was messing around, and got to work.
There's a nice, lightweight Linux package called
dnsmasq that will handle both DNS and DHCP. I found it super easy to set up on one of our Fedora servers, thanks to
Keith Fieldhouse's article on linux.com; the hardest part was typing in all the MAC addresses for servers that I wanted to have a fixed IP address, since the router's web UI wouldn't let me copy and paste the DHCP MAC to IP address mappings.
The only other tweak I had to make was to configure the network adapter on the dnsmasq machine to have a fixed IP address, rather than trying to get it from DHCP.
After a couple of hours of setup and careful testing, everything was up and running. An extra benefit of having a local DNS server is that it caches all its results, so that lookups to a host that it already knows about are nice and fast. In fact, I've now set up a local dnsmasq server just for DNS at home, since anything that makes web browsing faster has to be a good thing !