Wearing of a head covering (yarmulka, skullcaps, kippah [pl. kippot]) for men was only instituted in Talmudic times (approximately the second century CE). The first mention of it is in Tractate Shabbat, which discusses respect and fear of God. Some sources likened it to the High Priest who wore a hat (Mitznefet) to remind him something was always between him and God. Thus, wearing a kippah makes us all like the high priest and turns us into a "holy nation." The head covering is also a sign of humility for men, acknowledging what's "above" us (Gd).
Many Ashkenazi rabbis acknowledge that wearing a head covering at all times was once considered an optional midat chasidut (pious act), but, today, fulltime head covering is the norm except under extenuating circumstances. Sephardic communities generally did not have the custom of wearing a kippah all the time.
Some diaspora Jews leave off the kippah at school, work, or when testifying in court, because of real danger or uneasiness in appearing in the secular world with an obvious symbol of Jewishness.
In Israel wearing a kippah also has a social significance. While wearing a kippah shows that you are somewhat religious, notwearing one is like stating, "I'm not religious." The style of kippah in Israel can also indicate political and religious affiliations. Classical orthodoxy uses a large, smooth, black one shaped like a bowl. Many Hasidim use large black felt or satin, and a "rebellious son" may wear a slightly smaller black kippah to show his independence while remaining in the Classicist camp. Another play on this rebellion is to wear a knitted black kippah. This is also usually used to confuse people as to where you stand.
In Bukhara and the Caucasian Mountains the use of a large brightly woven kippah is common. It is similar in shape to a cantor's kippah without the peak. This custom can also be found in other Sephardic communities.
Knitted kippot typically signify that you are part of the Nationalist Zionist camp. A larger full headed knitted kippah would signify the Mercaz Harav branch of the movement that produces many of the leading rabbis within the Religious Zionist section, although many rabbis who teach at these institutions wear the traditional large black kippot.