The Eye-Tracking Study of the Film Frame Composition Influence on the Visual Perception

IntroductionCinema, as a cultural phenomenon with rather specific features, has been formed after going through many stages of its development.

The progress of cinema was not only in technical terms, which consisted in the development and improvement of the means of shooting and editing the film, but also in artistic. The element of creativity did not immediately enter into the direction of the film, it took a long time for the formation of cinema as an art object. Currently, we can talk about the presence in the cinema of its own, unique artistic language of narration, which has its own syntax ^[1].

The perception of a work of art, including a movie frame, is closely related to the physiology of human perception of visual information. The laws of frame construction in cinematography are similar to works of painting and strive for such a reflection of reality that would contribute to a deeper immersion of the viewer into the action taking place on the screen. An important role in the process of immersing the viewer in the atmosphere of the film is played by a properly and qualitatively designed film frame, which is the minimum structural unit of the film.

There are several alternative approaches to the pattern of information perception. One of them considers this process as integration by spatial comparison of information by summing information from successive fixations ^[2]. An alternative approach assumes that information from successive fixations is aggregated not by combining "snapshots" of fixations, but by integrating more complex visual attributes at medium and high levels of analysis ^[3].

The presented research is aimed at studying the elements that affect the visual perception of the film frame in order to develop methodological recommendations for the harmonization of the frame in the process of filmmaking.

The object of research is a movie frame. The subject of the research is the technology of film frame processing.

1. Theoretical modelCurrently, there is no generally accepted unit of measurement for the visual attractiveness of a movie frame, which does not allow comparing frames with each other by comparing their parameters.

A rather subjective expert analysis is used to assess the quality of the work performed.

By the visual attractiveness of a movie frame, we understand the property of a movie frame aimed at attracting and retaining the viewer's attention as a result of the combined impact of its elements (Fig.1). The compositional structure of the frame consists of three main elements. The first is the composition. The second element of the movie frame is the presence of movement. And the third is the color scheme ^[4].

The visual appeal of a movie frame is influenced by "informativeness" and compositional construction. In turn, the "informativeness" of the frame is determined by the director's idea, and the realization of the idea takes place with the help of a compositional structure consisting of three components – composition, dynamics and color scheme (Fig.1). As a result of an experimental study, it was revealed that the "informativeness" of the frame affects the pattern of viewing the movie frame ^[5].  The viewing pattern is understood as a set of quantitative parameters of oculomotor activity obtained using the eye-tracker software and hardware complex when the subject considers the stimulus material ^[6]. The compositional construction of the frame, as its integral component, also affects the viewing pattern. Analyzing the parameters of the viewing pattern by methods of mathematical statistics, it is possible to compare the visual attractiveness of frames used as stimulus material in the experiment.

Fig. 1. Frame design scheme. Frame elements affecting visual appeal

Within the framework of this study, a number of experiments were conducted, the purpose of which was to study the significance of the influence of the color scheme ^[7,8], the "informativeness" and stylization ^[9] of the movie frame on the perception of its viewer. The work of the visual system for image recognition occurs in three stages (Fig.2). The first stage is a mechanical scanning of the image with the eye. It occurs unconsciously and consists in a cursory "feeling" of the image with the eye. The second stage is the recognition of the image seen – the work of the brain is connected. The third stage is the emotional response of the viewer. A person makes his own impression of what he has seen. As a result of an experimental study, it was found that the time required for image scanning is a constant value, and the recognition time of the stimulus depends on the content.

Fig.2. The scheme of the human visual system for image recognition

The purpose of this work is to obtain experimental data of the movie frame viewing pattern and to identify statistical patterns to confirm or refute the formulated hypothesis.

Statement of the research problem. To conduct an experimental study of the effect of composition on the parameters of the movie frame viewing pattern.

2. Incentive materialFor the compilation of the visual series, stimulus material, selected frames from films.

The content of the frames met a number of requirements, such as: emotional neutrality and the presence of a minimum semantic load, but the objectivity should be preserved. In addition, the frames were selected in such a way that the image had two centers of interest having different mutual area relationships. As a result, a database of 120 personnel corresponding to the set conditions was created (Fig.3).

Fig. 3. Example of stimulus material

The purpose of this experiment was to study the influence of the composition factor on the perception ofyatiya kinokadra. In the described experiment, attention was paid to the compositional construction of the frame – the mutual ratio of the areas of the centers of interest to the background. The stimulus material of the experiment is selected so that the mutual ratio of the areas of the centers of interest to the background can be divided into three groups. As a result, images were obtained that can be unambiguously divided into three groups (Fig.4): objects occupy less than 25% of the frame area, from 25 to 40% and a group in which objects occupy more than 40% of the frame area. The percentage ratio of the sum of the areas of objects (centers of interest) to the background was calculated.

A) b) c)Fig.4. Stimulus material with the percentage of the area occupied by the centers of interest to the background A) Less than 25%; b) from 25 to 40%; c) more than 40%

The next factor is the distance between objects (centers of interest). Stimuli are divided into two groups: in the first group, objects are located in the frame so close that they tend to be perceived as one gestalt (Fig.5a). In the second group of stimuli, objects are located at a distance greater than the object itself, which allows them to be perceived as separate entities (Fig. 5b). The field of view is defined as "the number of degrees of the angle of view with stable fixation of the eye ^[10].  As the distance between objects increases, the involved area of the subject's field of vision increases, which requires him to carry out more significant movements of the gaze across the frame space to identify objects.

A) b)Fig. 5. Stimulus material with distance between objects (centers of interest):

A) smaller, b) larger; the larger of the two dimensions of the object

3. Setting up the experimentCurrently, research using the technology that fixes oculomotor activity is actively developing ^[11].

Researchers present the latest technologies for gaze tracking and gaze evaluation. Following recent technological advances and the advent of affordable AI trackers, there is a growing interest in widespread technologies, attention-oriented systems and interfaces that can revolutionize the generally accepted human-technological interaction ^[12].

To conduct the current experiment, a hardware complex was used that captures oculomotor activity – the SMI RED 250 AI tracking software and hardware complex (Fig. 6). A chair and a stand for fixing the head were set up for each subject. The I-tracker was also calibrated individually using a function built into the complex.

Fig. 6. Experimental setup with an AI tracker: computer monitor, AI tracker and headrest

In the present experiment, the subjects were asked to solve the problem of identifying the stimuli demonstrated to them at the first stage, among many others. The experiment recorded a change in the frame viewing pattern depending on two factors: the area of the main objects in the frame and the distance between the objects.The results of the experiment on the influence of the color scheme are described in the article^[4].

The experiment consisted of two stages. At the first stage, 10 stimuli were presented to the subject. It was necessary to memorize the images. There was no time limit for memorizing. After a time interval (30 min.), the subject was presented with 50 stimuli, which included all the frames from the first part of the experiment and randomly selected 40 stimuli from the database. At the second stage, the subject was asked to solve the task of recognizing the stimulus. Each participant of the experiment is asked to choose from 50 frames those stimuli that he has already seen before.  The order of demonstration of the material at both stages is sequential and random. The duration of the examination of the stimulus by the subject at the recognition stage was also set by the subject independently.

4. ResultsThe experiment involved 44 people from 18 to 25 years of age from students of Peter the Great St. Petersburg Polytechnic University.

Statistical processing of the experimental results was carried out by means of ANOVA analysis of variance ^[13]. The following parameters of the viewing pattern were analyzed: the time of observation of the stimulus, the average duration of fixations when observing one stimulus, the average number of fixations when observing one stimulus, the average time of saccades and the average number of saccades when observing one stimulus. The task of the analysis was to identify the influence of factors of the size of objects and their distance from each other. The value of the p-value significance criterion for hypothesis acceptance was 0.05. The p-value values obtained as a result of the computational procedure are presented in Table 1: the Square factor is the percentage ratio of the sum of the areas of objects to the background; the dist factor is the distance between objects.

Table 1. Values of the p-value significance criterion for the studied factors

A graph of the density of the distribution of the stimulus viewing time depending on the size factor is shown in Figure 7.

Fig. 7. Graph of the density of the distribution of the time of viewing the stimulus depending on the factor of the size of objects

Along the ordinate axis, the values of the parameter of the duration of viewing the stimulus in milliseconds are located. The values for stimuli divided into three groups by the factor of the mutual ratio of areas are presented: less than 25% - the size of objects is insignificant compared to the area of the frame, such objects are more difficult to see because they are lost in a large background space; 25-40% - it took the least amount of time for the subjects to consider this group of stimuli, from which it can be made the conclusion is that this group of stimuli turned out to be the most convenient for viewing; more than 40% - the size of objects is so significant that they tend to be perceived as a background, so it takes more time to identify them.

Experimental data have shown that the factor of distance of objects from each other affects the perception of the movie frame (Fig. 8).

Figure 8. Graph of the density of the distribution of the stimulus viewing time depending on the distance factor between objects: 1 – the distance between objects is less; 2 – the distance between objects is greater than the object

Along the ordinate axis, the values of the parameter of the duration of viewing the stimulus in milliseconds are located. The values for stimuli divided into two groups by the distance factor between objects are presented: 1 – the distance between objects is less than the object; 2 – the distance between objects is greater than the object, so the subjects needed more time to view a larger area of the frame.

Based on the results of statistical analysis, it can be argued that the parameters of the viewing pattern have a statistically significant dependence on the factors of the size and distance of objects from each other in the frame.

5. ConclusionsWhen analyzing data on the influence of the size factor of objects, the results obtained show that under the condition that objects occupy a small area of the frame, the observer needs more time to consider this frame.

As well as in the case when objects occupy most of the frame (more than 40%). It can be assumed that, in the first case, due to the small size of objects, it becomes more difficult for the observer to find objects in the frame space, and in the second case, it takes time to identify objects, since they tend to be perceived as a background due to their large size.

If we consider the factor of the distance of objects from each other, it is worth noting that with a greater distance between objects, the subject has to shift his gaze over the area of the frame, which makes him look at the frame longer.

The composition affects the pattern of viewing the frame by a person. When two objects are located at a greater distance from each other in the frame, the viewer considers such a frame longer.

As a result of the work done, a methodology has been developed for conducting experimental studies of human perception of visual information using an AI-tracking software and hardware complex.

The developed technique was tested on 44 subjects from among the students of Peter the Great St. Petersburg Polytechnic University and showed its viability.

A movie frame is a conditional interface for transmitting visual information ^[14]. The developed technique can be used in conducting research on the perception of visual information in graphical interfaces of remote control systems for dynamic objects.