In order to select an appropriate power supply, several factors must be considered.
The first consideration in selecting a power supply is whether the board you're powering has a voltage regulator. If you designed the board, you should know :) If you're working with a dev board, the acceptable supply voltage should be specified in the documentation. Using mrbasher's board as an example, if the board has a regulator, it will usually be located near the power connector and look something like the photo below. Not all voltage regulators use this TO-220 package shown but it is quite common.
The next consideration is what voltage the parts on the board actually need. Many PIC circuits require 5 volts but 3.3 volts is becoming common too. If the board does not have an on-board regulator (the TAP-28 board is a prime example), we'll need a regulated power supply of the correct voltage. If there is a regulator on the board, the power supply will have to supply a voltage higher than the regulator voltage + the regulator dropout voltage. Some details follow.
The next consideration is how much current the board draws. This is the current to power the chips, light the LEDs, power anything attached to the board and to support any pull-up or pull-down resistors on the board. LEDs and other things attached to the board are probably the biggest draw. Still using mrbasher's board as an example, the resistors look to be 270 ohms, and a typical green LED has a voltage drop of 2.1 volts, so
We'll call it 11 mA per LED x 8 LEDs = 88 mA when all the LEDs are illuminated. The current for the PIC and the 74HC595 shift register won't add much, so our power budget is around 100 mA.
Our power supply has to supply at least 100 mA, but more current is ok.. The circuit draws only the current it needs to operate. If the power supply is rated for say 1000 mA, it's going to be perfectly happy loafing along supplying 100 mA or less.
Back to the question of what voltage is needed. Assume the voltage regulator is an LM7805. The dropout voltage for a LM7805 is 2 volts. Think of this as the overhead it needs to work. The maximum voltage an LM7805 can handle is 35 volts. So any voltage between 7 volts (Vreg + Vdropout) and 35 volts (the maximum voltage) will provide a 5 volt output from the regulator. Piece of cake.... Unfortunately, it's not quite this simple. LDOs (low dropout) regulators are available that reduce the needed overhead voltage.
The LM7805 is a linear regulator (the following applies to all linear regulators). The current into the regulator is the same as the current out of the regulator. Our circuit uses about 500 mW maximum (5 volts x 100 mA = 500 mW). If we supply 35 volts to the regulator, a total of 30 volts is dropped or lost across the regulator. 30 volts x 100 mA = 3 watts. Where does this energy go? Heat. Have you ever burned your finger on a night light bulb? They draw 4 watts. Our regulator isn't going to be too happy dissipating this much power.
The voltage supplied to the regulator should be close to the voltage it needs to work to minimize power dissipation. What happens if we supply it with 9 volts? The drop across the regulator is 4 volts, so the power is 400 mW. This change drops the power dissipation by a factor of 10 and now our regulator is happy. How high a voltage level is acceptable depends on the current draw of the circuit. If the current draw is high, the voltage should be close to the voltage needed by the regulator. If the current draw is low, there's more latitude. The maximum allowable voltage depends on the actual regulator used. The equation below is the power dissipated by the regulator:
Current must be greater than the maximum draw of the circuit being powered but anything larger is ok
Voltage for a board without a regulator must be regulated at the voltage required
Voltage for a board with a regulator must be greater than Vreg + Vdropout but not too much higher