Some capacitors ARE polarity conscious. Others are not. If there is a Plus Sign "+" or, a minus "-" sign then polarity does matter. These capacitors can be very small to very large. Tantalum capacitors are used very frequently because they have a very high "capacity", measured in small portions of a farad, in a small size. If your capacitor is in a metal can, it is almost certainly polarity conscious.
Other capacitors may look like a small button with two wires coming out the bottom (usually). These are not polarity conscious.
Remember also that capacitors are rated to only be usable up to their rated voltage. Exceeding this voltage, or installing a polarity conscious capacitor incorrectly can actually cause the capacitor to explode.
Pictured are two types of polarized capacitors an electrolytic and a Tantalum. Please note that tantalum caps can come in many shapes, colors and sizes.
Here is a sample of a non-polarized capacitor