Judaism can be thought of as being simultaneously a religion, a nationality and a culture.
Throughout the middle ages and into the 20th century, most of the European world agreed that Jews constituted a distinct nation. This concept of nation does not require that a nation have either a territory nor a government, but rather, it identifies, as a nation any distinct group of people with a common language and culture. Only in the 19th century did it become common to assume that each nation should have its own distinct government; this is the political philosophy of nationalism. In fact, Jews had a remarkable degree of self-government until the 19th century. So long as Jews lived in their ghettos, they were allowed to collect their own taxes, run their own courts, and otherwise behave as citizens of a landless and distinctly second-class Jewish nation.
Of course, Judaism is a religion, and it is this religion that forms the central element of the Jewish culture that binds Jews together as a nation. It is the religion that defines foods as being kosher and non-kosher, and this underlies Jewish cuisine. It is the religion that sets the calendar of Jewish feast and fast days, and it is the religion that has preserved the Hebrew language.
Is Judaism an ethnicity? In short, not any more. Although Judaism arose out of a single ethnicity in the Middle East, there have always been conversions into and out of the religion. Thus, there are those who may have been ethnically part of the original group who are no longer part of Judaism, and those of other ethnic groups who have converted into Judaism.
If you are referring to a nation in the sense of race, Judaism is not a nation. People are free to convert into Judaism; once converted, they are considered the same as if they were born Jewish. This is not true for a race.
Sources: soc.culture.jewish FAQ