2.1 Computer Graphics Model

This paper presents a methodology to evaluate images aimed at achieving photorealism, that is, simulating the appearance of a real photograph. In previous studies, we have developed image representations of plasma particles to check for simulation correctness, for example the Poincaré plots and magnetic fields maps [⁵]. In this work however, our interest lies not in evaluating images used to check for simulation correctness but in using and evaluating images whose main function is broader science communication. Meaning, communicating the results of an investigation to the interested community or gaining the attention of a non-technical/scientific audience.

After defining the images to be evaluated, the need arose to design a ray tracing model capable of generating high-quality photorealistic images that would give the observer a more accurate idea of how the stellarator-confined plasma looks. A previous study [²] had presented a model that generates images from the BS-SOLCTRA results.

This model uses the simulation results to convert the raw data into a mesh that represents the shape of the plasma and once that mesh is reconstructed it can be inserted as an object within the scene of any renderer. The results of the simulation consist of a set of files where each one represents all the steps that a particle had during the whole simulation. Each file uses comma-separated values to represent the 3 dimensional particle position at each iteration step. What is important is the selection of data necessary to successfully reconstruct the surface mesh.

It is thanks to Poincaré plots that we know that plasma surfaces are formed by the trajectory of a single particle. To build the desired surface it will be enough to find a suitable file. BS-SOLCTRA in its original version generates the position data of the particles that will form a surface, so the decision was to choose the largest file since it has the largest amount of data. Once the largest file is found it is converted to an 〈x,y,z〉 file format to be used as input by the surface reconstruction algorithm.

2.1.2 Lighting and Density Model

Once the extracted surface is added into the scene, the next step for the model is to add visual attributes to that object so that it looks similar to plasma. To get a better idea of what plasma should look like, real photos of confined plasma from different confinement chambers such as stellarators were analyzed. These photographs give an idea of the physical characteristics of the plasma, being like a gas suspended in the vacuum of the chamber with bright flashes of a color given by the gas used in the discharge.

The plasma seen in a real picture is similar to a gas and with a not much density, so what is behind from it can be seen, similar to what occurs to a translucent object. Another interesting attribute that can be seen in that last surface, is the light scattering happening in the surface, simulating its own light emission in blue and purple tones.

Figure 1 will be a guide to a better understanding of the effects of parameter changes in the final result. The cube will represent the plasma material and the red sphere will represent whatever is behind the plasma in the scene, since our goal is to see what is behind the plasma due to its translucency.

Fig. 1 Plasma model differences in diffuse illumination, density and luminosity attributes 

For the model creation, the work is based on modifying different parameters for each object to be described into the scene, these parameters will help to calculate both the surface and its interaction with light. Three of these parameters are called ambient, diffuse and specular lighting, referencing the Phong lighting model [⁸] used in ParaView. The three parameters are represented with numerical values between 0 and 1 to describe the contribution of each to illumination on the object surface. Those attributes will let us describe the light emission effect of the plasma by scattering the incoming light on the surface.

Ambient lighting brings a dim light to objects in the scene, simulating interaction with light from a very distant source, adding a bit of color to the object even without nearby lighting. The diffuse lighting simulates the directional impact that light has on an object, being the most significant visual component of the lighting model. The higher its value, the greater object visibility and brighter colors. Specular lighting is the light that reflects bright objects coming from the light source so the color is more associated with the light source than with the color of the object.

In our case, the specular reflection will be 0 because we don’t want any light source reflection in our plasma. We only want to reflect its own bright. Also the ambient light will not be needed because we only need the effect of the diffuse component combined with the opacity value and color. As it is shown in figures 1a, 1b, and 1c, we can notice more object light emission as we set a higher diffuse lighting value.

There are two attributes related to the object that will give the final appearance that we are looking for in the plasma, the opacity and luminosity of the object. The opacity of the object is a very important attribute for our representation since it is in charge of giving the effect of a translucent object and a little dense gaseous texture described above. The lower opacity value within our model, the lower the density of the object, so we define a value between 0.01 and 0.4 for a suitable translucent representation of plasma. In figures 1d, 1e, and 1f, the appearance is changed as we increment the opacity of an object.

On the other hand, any value greater than 0 in the luminosity of the object turns the object into a light emitter black body. The issue with this attribute is that this kind of objects lose their translucency and the result would lose an important element of the realistic appearance. This effect can be used to obtain an unrealistic but a different light-emitting representation of the plasma that can be used to catch the attention of the audience or similar cases. Figures 1g, 1h, and 1i show how the luminosity affects the material translucency and how a value higher than 0 in the luminosity affects how we see through the object.

3.1 Qualitative Questionnaire

Evaluating these images should be done cautiously, given the amount of subjectivity involved in this process. Each evaluator could provide varying opinions about an image, making the process of coming to conclusions difficult.

The way of comparing them at a qualitative stage is a lot simpler and handy to our possibilities and it grants a scale based on the concept called Visual Plausibility on which the evaluators can rate each image. In this way, we can diminish the subjectivity of the evaluation.

For this matter, the evaluation tool needed must be capable of collecting and documenting information regarding our evaluators, like their knowledge, experiences and their backgrounds. On the other hand, it must provide a way so that any person could rate any presented image and also be capable of summarizing the results of the image evaluation.

As [⁴] mentioned, the answers to qualitative questionnaires consist of memories, opinions and experiences. This kind of questionnaires generate a rich material, useful for researchers from many disciplines.

This provided material is highly informative about various aspects of everyday life, and depending of how the question formulation is, the researchers are able to subtract any kind of information from the evaluators.

The method used is a qualitative questionnaire, which consists of collecting memories, opinions and experiences for a specific situation. Respondents answer the questions in the questionnaire based on these memories and experiences to give an opinion on what was asked. So our task is to correctly design the questions to collect their knowledge about the appearance of the plasma and the evaluations of the plasma model without interfering with their opinion.

According to [⁴] the use of qualitative questionnaires to produce research material has been criticized, not least because of the mentioned lack of representation from all social strata amongst the respondents. But, it is important to keep in mind that the material the tool generates can form the basis for generalization.

As with other qualitative methods, its strength lies in the deep insights that may be gained from the respondents. Such is the case in this study, where we are interested in the expert criteria to evaluate with their opinions our results, not to get a representation from all kind of people in the society.

3.2 Question Design

To develop the survey questions, different factors associated with it must be taken into account. Among the factors is the number of people, who to direct it to, number of questions, information you want to collect about the respondents, in addition to defining which images will be evaluated.

The goal about the amount of respondents was a number between 15 and 20 by recommendation of expert colleagues in the field of information of visualization. Regarding the target audience, the intention is to direct it to experts on plasma physics and even better if the person has had experience seeing plasma either physically or in a photograph.

These people are the right ones to evaluate a photograph that aims to render photorealistic images due to their previous knowledge and experiences added to the concept of visual plausibility created for the evaluation of the images.

The questionnaire was designed in such a way that with 5 questions before evaluating the images, it can be avoided that the respondents do not understand the concept of visual plausibility designed to evaluate and, on the other hand, it can be ensured that they are the appropriate people to make the evaluation. The first question aims to classify our evaluators in their relation with the plasma laboratory. This is why the first question was What is your relation with plasma physics?

The second and third question try to shed light on the respondents’ context and experiences with plasma. The second question asks if they have worked with high temperature plasma discharges either directly or indirectly. The third question asks if they had ever seen a real picture of plasma confinement in devices like stellarators or tokamaks.

There is a fourth question that we asked to our evaluators. It tries to answer how they think the plasma should look like before starting to evaluate our images. In this way, we can get an idea of their expectations before we show a real image.

The fifth question was designed to ensure that the evaluators understand the concept of visual plausibility. We asked if they already know the concept of “plausibility” and remind them that if they are not familiarized with it, they can go back and read the survey introduction where it is defined.

Visual plausibility aims to recognize that although something isn’t real, it looks very similar to how it looks in real life. In order to evaluate our images, we propose a scale from 1 to 10 to evaluate the visual plausibility of the picture, 1 being a picture with total lack of visual plausibility and 10 being a picture that looks very similar to a real one.

The first picture (Figure 2) to evaluate was rendered using PBRT in order to model plasma appearance using a similar approach to the result of this work. We extracted the surface from the particles and the object was the input for the ray tracer, this time to give that surface a plasma appearance we had to create a mixed material using glass material and scattering surface material. The result was not so satisfactory but an interesting approach that although not very realistic is worth evaluating in order to confirm that respondents can differentiate a representation with low visual plausibility from one with high visual plausibility.

Fig. 2 First picture in evaluation 

The second picture (Figure 3) to be evaluated aims to do the same as the previous one, to show the respondent a plasma representation that is not intended to be realistic, so low ratings are expected for both this and the first representation. For the rendering of the image, Paraview[¹] was used as software and the approach was to render the data of a simulation that calculated the trajectory of a million particles.

Fig. 3 Second picture in evaluation 

In Figure 4 we can see a variant of our solution. The idea was to make this model as flexible as possible and we found out a way to represent a black-body radiant object, so the plasma can irradiate light without the external light source.

Fig. 4 Third picture in evaluation 

This definitely was an attribute we wanted to have in our solution but the problem is that the ray tracer shows converts the object into a black body one and this means that it doesn’t show any transparency at all, actually we cannot see through the volume, and this transparency is an important attribute that we definitely want to have into our final model.

Just as we mentioned with the last two approaches that we didn’t intend to be realistic or get a good score with these representation, this image doesn’t pretended either. The intention is to try to evaluate different approaches that we face throughout the process of this work and that may be useful on other occasions.

To render this image, the luminosity model described in previous sections was used, which converts the object into a blackbody object that emits light. This is a way of representing plasma from a different perspective that is intended to impress the viewer but is unrealistic so a high score is not expected.

The last two images (Figure 5) are equivalent to the ones used in the questionnaire. These images differ slightly from the ones actually used, in the first one what changes from the one used in the survey is the angle from which the camera is positioned to obtain the image. On the second image, what changes is the color used, initially blue tones were used for testing and the final rendering was done in pink-purple tones according to the tones emitted by the gases used in reality.

Fig. 5 Final photorealistic plasma model (a), and final plasma surface in vacuum (b), both rendered with OSPRay 

These two images were obtained using our model, so that the evaluation of these images represents the evaluation of the proposed model. Figure 5a considers details such as the actual material of the stellarator device or at least a similar one, the density of the plasma, the translucency and the exact color according to the gas used in the plasma discharges.

And for the fifth image we thought that the chamber may confuse or distract some people from evaluating only the plasma appearance, so we added one more image from the pipeline without the chamber to completely appreciate the plasma shape, the emptiness of the volume and the color of the low density gas. Figure 5b shows a very similar representation of the image displayed in the last image evaluation question differing only in its color.

4.1 Respondents Evaluation

The intention with the first question was to give us an idea about our image evaluators, so that we could make sure that a vast majority are involved with plasma physics work and that they at least have knowledge on the subject. That is why the first question was What is your relationship to plasma physics? The result shows that half of the respondents are researchers from the plasma physics laboratory. The other half of them are divided between students and research assistants. Table 1 shows the results obtained for the first question.

The second and third questions help in understanding the context and experience of the respondents. The second question is whether they have worked with high-temperature plasma discharges, either directly or indirectly. The answer to this questions reveals how much our respondents know, since 17 out of 18 people answered that they have worked with plasma discharges, at high temperature directly or indirectly as we can see on Table 2.

The third question is a continuation of the previous question as it asks if they have ever seen a real picture of plasma confinement on devices like stellarators. Table 3 shows how 88.9% (16) have seen a real image before. That shows the experience of our testers, they have worked with plasma and they already know what a real image looks like, so they are the right evaluators to rate the images generated by the model by giving you a grade on their visual plausibility.

There is a fourth question we asked our testers, it tries to answer how they think the plasma should look before starting to test, this way we can get an idea of how they see the plasma before we show them a real image. The results confirm assumptions made from the beginning when we analyze the photographs of plasma discharges from different devices.

The responses have many matches between them, with words such as gas, bright, bright, region, shape, fuchsia, light, gradient, gaseous, flow, glow, light, luminous, intensity being among the majority of responses written by respondents. This shows that the initial idea of what the plasma would look like is not far from the experts conception of the same idea.

Finally with the fifth question we wanted to make sure that our respondents knew the concept designed to evaluate the images. So we asked them if they already knew the concept of ”plausibility” and reminded them that if they are not familiar with the concept, go back and read the introduction to the survey where the definition of visual plausibility was found. The table 4 shows the results and illustrates that half of the respondents, that is, 9 people, did not know the concept previously and that 2 more were not sure if they knew it or not.

4.2 Image Evaluation

This part of the evaluation is the one that finally gave us the opinion of the expert criteria on the work we are presenting. Each expert evaluator rated from 1 to 10 the visual plausibility attribute of 5 different images shown to them. Our idea in this section was to compare the images generated by our work with images generated with other tools which did not present much realism since that was not the intention when they were generated. In this way, by comparing unrealistic representations with others that are much more realistic, we were sure that the concept of visual plausibility was understood and that although the images in the model are not hyper-realistic pictures, they do simulate a simple photograph with high-quality results.

For the first image of the evaluation we use an image rendered with a ray tracer called PBRT. This image is the result of a proof of concept to make the photorealistic model, which in our project was a dead end since the results did not meet the needs of the study. Still in this way the data obtained for the first image were shown in Table 5. The result of the evaluation of these approach was a low visual plausibility. With a mean of 5.65 and a standard deviation of 1.84 we got a non realistic result but still interesting because a lot of evaluators appreciate the translucency that the glassy material achieved and that glitter that plasma irradiates.

As we can see in Table 5 the 77,7% of the answers do not consider this approach with high visual plausibility. These results show how bad was the results using PBRT since these image was the most accurate representation we achieve using that ray tracer (mean score x=5.65, standard deviation s=1.84).

As we expected this representation was the lowest scored with a 4.2 mean for visual plausibility rate and a standard deviation of 2.33. It is important to remember that it lacks of reality because it does not tries to be a realistic representation. It shows a very accurate shape but without the translucency we are looking for, it does not looks like a gas or with low density volume, nor shows luminosity either. However, interestingly in Table 6 we can see how we have a result of 10 which is clearly an outlier which affects significantly to the calculation of the mean and the standard deviation (mean score x=3.8, standard deviation s=1.99).

Table 7 shows the results of the evaluation of the third figure. As the table shows, with a mean of 5.55 and a standard deviation of 1.87 we had a 88.9% of the evaluators that considers the image has low visual plausibility. Although the image is not a realistic one, they see the value in this representation for showing results and cause a good impression in the audience (mean score x=5.55, standard deviation s=1.88).

As showed in Table 8 with a mean of 8 and a standard deviation of 0.94 we had an impressive result of the 72.2% of the answers considers it with high visual plausibility. This is the second highest score we got from our evaluators and the first evaluation for our final resulting model. These results were obtained by filtering the data, eliminating those outliers that, as in the second question, significantly modified the calculated data.

We can see how the mean value is around 8, however a vote of 1 and another one of 2 negatively affect the rating, which could be due an evaluation misunderstanding. The value calculations counting the outliers was lower (mean score x=7.28, standard deviation s=2.30) than without those outliers answers (mean score x=8, standard deviation s=0.94).

The satisfactory results shows the highest score in the evaluation. With a mean of 7.94 and a standard deviation of 0.99, we got the same 72.2% of the evaluators who consider it has high visual plausiblity. Using the same logic of the fourth image evaluation, we can see in Table 9 how a vote of 1 affects the rating mean decreasing it and increases the standard deviation, which could be due a evaluation misunderstanding.

The value calculations counting the outlier were lower (mean score x=7.58, standard deviation s=1.87) than without the outlier data (mean score x=7.94, standard deviation s=0.99).

5.1 Conclusions

To make particle fusion a productive energy source, it is imperative to continue developing simulation and visualization tools that help scientists and engineers build efficient fusion devices. Scientific visualization has a twofold contribution.

First, it provides researchers with a powerful tool to deeply study complex phenomena. In the case of particle fusion, visualizations help in understanding the behavior of variables of interest. Second, scientific visualizations provide a communication tool for a broader audience.

Having people well informed about scientific endeavors is key in sustaining the public investment on groundbreaking research.

The result of this work presents a contribution in the way of evaluating works where the result is a series of images that must be qualified qualitatively. This evaluation method is very similar to the one used in information visualization contexts, where visualizations are made and opportunities for improvement are discussed at a qualitative level.

The results of this work were evaluated under a design methodology that exposes how much experience the respondents had working with plasma and if they had had contact with plasma images, which assured us that they were the appropriate experts to evaluate our images. The results obtained confirm that they comply with the proposed hypothesis, so the images generated by our model obtained a high visual plausibility score according to the expert criteria.

5.2 Future Work

Using this work as a starting point for future work, we consider that different efforts can be made to make the visualization tool much more realistic or useful for error checking. New visualization tools could be adopted, such as in-situ visualization to obtain high-quality images while the simulation is running. The possibility of generating frames of these visualizations and producing animations could also be explored.

We also consider a qualitative improvement could happen by adding a functionality in the ray tracer used, OSPRay in our case, so that the luminosity attribute can be used mixed with the opacity attribute of the object.

Hence, it would be possible to represent the object as a light emitter object but that it is not a black-body object. In other words, it would emit light without losing the characteristic of being translucent. By achieving this form of representation effectively, the physical aspect of the plasma can be represented in a much more realistic way.