Network design tips

Just thought I’d share my ideologies regarding the setup and management of a computer network. This is only a small piece of what’s required, and what I’m attempting to cover is common issues that are missed in company computer systems. This is targeted around a greenfield setup, however a lot of this should be able to be applied to an existing system.

Password Management

Storing backdoor passwords is always an issue. Eventually one day your going to need to get into the device when authentication services are down. A few common mistakes are made when it comes to backdoor passwords:-

  • Using the same username and password for every device or using an easy to work out pattern.
    • The issue with having a single username and password for all devices is that the password only has to be leaked once, and all your devices are compromised. A better solution is having a password database and giving each device a randomly generated password.
    • Some admins will suggest on a password pattern for things like wireless client authentication. This is very dangerous if a device is lost or stolen. If one of those passwords is lost, it’s very easy for an attacker to guess the pattern.
  • Storing the backdoor passwords in plaintext in a non-audited fashion.
    • If someone can just wiretap your communications to acquire your all your passwords, you.re doing it wrong. It also helps if the database records password accesses in case of a security incident.
  • All backdoor passwords shared with all admins.
    • It’s useless separating admin groups out in terms of what admins are allowed to access and what there not, if the backdoor passwords are accessible by all the admins. The server team only need server related passwords, not network related passwords.

It’s recommended that devices which support LDAP, Radius or other network authentication methods are configured for this. Backdoor passwords should only be used if something is extremely broken.

DNS structure

DNS can indeed up a lovely mess. In most part DNS should be carefully planned and documented. It’s very easy to look past DNS and just simply chuck everything into domain. This gets really messy and hard to work with.

First some things not to do:-

  • Don’t throw large amount of devices into a single zone.
    • If you have over 500 devices in a zone then surely you can find a way to cut them down into more manageable zones.
  • Don’t spend weeks on a host naming scheme
    • All it has to be is unique - all the information like criticality and server specs can be stored in as TXT record or in your device management system.
  • Don’t name servers after their purpose.
    • The server may change its purpose in the future, and then you.re stuck with an oddly named server.
    • Do however CNAME the service they are responsible for to the server’s hostname.
    • For example, don’t name the file server filer - label it waffles or E45629D00, then create a CNAME for filer to point to waffles.
    • If you’re having trouble working out what’s a good name for devices read RFC1178
    • Make sure you point clients at the CNAME and not the actual hostname.
    • It allows you to easily migrate services between servers.
  • Never have a standard that makes use of preceding zeros. For example, filer01.
    • The problem with preceding zeros is that there is a possibility that you’ll run out of device names at 100 devices. At this point you end up breaking your own standard.

Now some things that you should consider doing or planning: -

  • If you have multiple physical sites consider breaking them into their own zones.
    • I like this system - street.city.state.qld.au.domainname
  • Always start with the least common variable and finish with the most common variable last.
  • Physically label devices with the full hostname - not just the short name.
  • Make use of search domains.
    • Search domains can be really powerful and can help users reach the closest server.
    • For example a mobile user may move from one search domain to another. The new search domain would allow him to find the closest print server provided he used the short name and not the FQDN.
  • DNS is very flexible, and can become your entire infrastructure database. All it takes is a few lines of bash scripting and you easily store and query and information about a server in DNS. For example:
    • For server called testbox.mwheeler.org I could create subdomains like below to hold information regarding it.
      • cpus.testbox.mwheeler.org TXT 2
      • clock.testbox.mwheeler.org TXT 1.8GHz
      • locatian.testbox.mwheeler.org TXT Canada
      • prod.testbox.mwheeler.org TXT false
  • When you end up with the need to have multiple services under the same name ( eg multiple file server clusters), don’t name them filer1 filer2 filer3 in DNS. Label them 1.filer, 2.filer. You can also keep your original file CNAME and point it to 1.filer.

Network Topology and security

Network topology and security needs to be planned to ensure it meets the requirements of the business, and have the ability to expand to the requirements of the business in 5 years. time.

Routing is very important to this as it defines a core to your network where routing occurs. This prevents broadcast traffic from reaching all devices and helps isolate the network from issues that could bring the whole network down. It’s important to establish these routed boundaries early on, and plan communication for projects around them. A routed network makes firewalling easier to complete nicely.

Try to keep spanning tree topologies small in size. It helps with convergence time. Use modern spanning trees like MSTP or RSTP to speed up convergence.

In terms of general network security, common things such as BPDU guard and filter are left loff switches, MAC address limiting isn’t turned on, DHCP snooping isn’t configured or switch negotiation is still turned on. All these little issues create large security issues in your network, however are usually over looked. Monitoring of security logs is sometimes over looked, giving attacks the ability to brute force their way in.

When setting up firewalls try to group objects together to make the rules general. Managing a firewall with over 2000 ACLs easily becomes a nightmare to manage. Make the firewall rules as strict as possible. No point having a firewall rule if every device added to the network is allowed to talk to everything.

For large networks consider using GVRP or VTP. They have their own security considerations to look at, but they make managing the network a lot easier.

Never have an unrouted management network that spans all devices. This is a ticking time bomb for flooding.

Never set switch ports at a static speed and duplex unless the end device really requires it. See this post on why not.

Stay away from vendor specific protocols / systems. It’ll annoy you in the long run when you can’t replace out any of your switches because you’re tied to that platform.

Documentation

I’ve used numerous IP address management systems, ranging from custom built, to excel spread sheets to commercial software. All have their good bad and ugly features. First of all, don’t use a spread sheet. It’s hard to maintain, search and update. Secondly, don’t store your IP addresses in a database as strings. They are actually 32bits so they can be stored nicely as a LONG, and it makes querying the database so much simpler. If your admin team deals a lot with odd sized subnets, consider using something that visualizes the subnets - eg dokuipmap.

Use a wiki for documentation, but don’t forget about it. Wiki’s are a decent way to store data as they have an audit trail, easy to maintain and easy to back up, however they have to be used in a structured formal way. Make up templates, tags and groups and use them. Try to stay away from WYSIWYG editors as they mess up the syntax. Try to keep content as text rather than pictures. Try to use a wiki system that uses a file backend so it’s easy to backup and use offline during an outage.

Data and information duplication causes all sorts of problems when databases get out of sync. Try to keep the majority of your information for managing your network in one place, and where not possible, just have a single primary key - the device name. An admin shouldn’t have to update 4 different databases when he adds a new device to the network. If you have to enter data into to multiple places, automate it. Setup push/pull syncing of data between applications.

Have templates for device configuration. Copying other devices on the network leads to headaches when attempting to keep or update a standard.