This site uses the following assumptions to rate films:
Most readers will find this premise to be rather obvious. There are great films, films that are good but not great, mediocre films, and poor films.
A person cannot use a timer or a photometer to determine a film's quality. A count of the number of characters or killings would also be useless in determining the quality of a film.
The best way to tell if a film is great is to see if people who saw the film consider it to be great. People are more likely to consider a great film to be great than a poor film to be great.
When I was in teacher training, I was told that the difference between an average teacher and a good teacher is that a good teacher inspires one more student per classroom than an average teacher does. A teacher who is only average will still inspire some of the students, and a great teacher will still fail to inspire other students. Although I am not sure that one is the exact number, I believe that this can be applied to film. A good film may touch one more patron per showing than an average film touches. Great films will leave some patrons cold and mediocre films will still touch some.
This occurs because many factors influence each person's opinion about the quality of a film. The person's background, individual tastes, mood when watching the film, mood when asked to think about the quality of a film, the medium on which the person saw the film (e.g., theater, DVD, videotape, commercial television), comfort of the surroundings while watching the film, etc. all affect how someone rates a film. For example, I would guess that I saw most of the 3,000+ films that I have seen in my life only on a television. However, I have seen eight of my top nine films and 19 of my top 25 films on a big screen. It is possible that I would rate some of the rest a bit higher if I saw them on a big screen. No single rating of films can avoid factors like this.
Anyone who has taken an applied statistics class in college should know that the effect of random factors that affect a single observation can be reduced by making many observations. If I asked a single person what she thought of Citizen Kane (1941), it is possible that a poor mood or a dislike of black-and-white films or the poor quality of the print might have soured her opinion, regardless of the inherent quality of Citizen Kane. If I polled 19 other people who saw Citizen Kane in different circumstances, it is unlikely that all of them would be in the same sour mood, have the same distaste for black-and-white films, and have seen the same print. The first person would not stand out so much in the larger crowd. This is why public opinion polls have such large samples.
Even a single large sample poll can be problematic. Most polls of film quality draw from a homogenous sample. Some polls look at opinions of critics; other polls look at the opinions of average filmgoers. These two types of polls are likely to give very different results. Therefore, the aggregated results of several different polls would provide better results than any one poll would. This does not even address factors inherent to polls that affect ratings of quality, such as how many people have seen a particular film.
Psychologists have run into similar problems when measuring abstract concepts like personality or attitudes. Many of the measures that they used gave results that were close to meaningless. Some psychologists even questioned whether people even held attitudes or whether there was such a thing as personality. Eventually, psychologists figured out that measuring abstract characteristics with only a single measure is not very accurate. The best way to do so is to measure the characteristic using several different methods and using each method several times each.
The Phi Phenomenon uses many different types of lists to measure film quality. There are polls of the general public, of academics, of critics, and of filmmakers. Some of these polls ask respondents to list their favorite films. Others ask respondents to rate a large number of films. There are single author lists by critics, academics, and filmmakers. There are lists that focus on how films are rated by video guides or on how many awards the films received. Some lists try to call attention to obscure films. Others stick to the obvious choices. Some attempt to measure film quality directly. Others include factors such as the historical importance of the film.
The use of different types of lists and the statistical methods used help to minimize the effect of factors other than quality in order to create a purer measure of a film's greatness. It is not perfect, but it is closer to perfect than any other method is.
Imagine a group of ice cream experts. They know a lot about ice cream and are very good at determining which brands of ice cream are great, which are merely good, and which are poor. However, they do not agree on everything. Some might prefer chocolate ice cream, others might prefer strawberry, and still others may prefer vanilla. Even if there is an objective way to measure their opinions about an ice cream's greatness, a chocolate fan might still rate the fifth best chocolate ice cream as being better than the second best strawberry ice cream, whereas a strawberry fan may do the reverse.
People have different tastes in films. Some like action films; others prefer romantic comedies. Statistical procedures can be used to look for patterns in ratings to find groups of lists that are similar to each other. There appear to be three well-defined groups, which are called "tastes." These tastes are one reason why individual lists of great films differ from each other.
The tastes in films are more complicated than the tastes in ice cream. One could point out that people with highbrow tastes like films in languages other than English more than others do, and people with popular tastes like genre films (e.g., fantasy, science fiction, thriller, and horror films) more than others do. There are many exceptions. A Mandarin-language film, Crouching Tiger, Hidden Dragon (2000), is a highly ranked popular film. Genre films such as 2001: A Space Odyssey (1968) and Vertigo (1958) are highly ranked highbrow films.