The blues and reds are more evenly distributed, so local neighborhood recovery is less dependent on the area of the visualization than global neighborhood recovery. Again, Local MDS performed badly, indicating that the neighborhood size was perhaps set too low. The diffusion maps technique has the best overall neighborhood recovery, which demonstrates that a technique can have trouble recovering an overall shape, but can still have good neighborhood recovery.
The process was repeated for the Swiss roll dataset. The all k visualization for the Swiss Roll is given in Figure 5. The Swiss roll is quite often used as a classic example of how global dimensionality reduction techniques can have problems with local neighborhood recovery, as the distances between the concentric layers of the roll can distort the results from dimensionality reduction methods.
All the other methods had poor overall recovery. The kernel PCA vs. PCA comparison plot has almost disappeared, as the configurations have almost identical item-wise agreement, making most of the points the same color as the background. The poor performance for PCA may be due to the width dimension of the Swiss roll collapsing onto a two dimensional spiral. This means that items are almost randomly placed on top of one another, giving poor item-wise local recovery. It would be possible to create a series of scatterplots that covered different values of k, but for item datasets, multiple values of k must be combined in a scatterplot.
Heatmaps have long been used to analyze values over sets of parameters [61] or for comparing quality values across techniques. For example, [57] give a polygon heatmap for three different solution quality metrics stress, correlation, and neighborhood preservation , where each technique represents a vertex of the polygon and the points in the interior of the polygon are determined by a weighted sum of the values at the vertices.
Heatmaps have been used to analyze dimensionality reductions solutions, for example, by relating source dimensions to the lower dimensional projection plane [76].
In the QVisVis framework, a heatmap can display agreement over both k and item dimensions. The first graph, given in Figure 7, shows a comparative heatmap, showing the results of Kernel PCA vs. The overall agreement across all k is 0. One can see that there is a good deal of consistency across k.
Most of the items in the center of the configuration have outperformance with Kernel PCA over the whole range of k. To give a sharper idea of wins vs. This binary map is given in Figure 8. The Heatmap for LLE vs. Smacof on the large diameter regular torus is an interesting one.
In terms of overall agreement Smacof dominates LLE quite substantially. However, the techniques are quite evenly matched for low values of k. But, as k increases, Smacof begins to dominate LLE on all items. This has been done to give a gradient map of partial stress [70]. It is also utilized in the VisCoDeR [18] tool for comparing dimensionality reduction algorithms.
In VisCoDeR, dimensionality projections are animated throughout the process of dimensionality reduction and are points are colored by data cluster. A greyscale Voronoi diagram that is shaded by local solution quality is displayed behind the projected points.
In [43], a similar Voronoi diagram is utilized, but tears and false neighborhoods in the configuration are given different colors. QVisVis utilizes a local regression and point interpolation loess [15, 72] approach to overlaying heatmaps onto scatterplots.
To ensure that the point colors do not blend into the loess colored background, points are given a black outline. In addition, points can be superimposed onto the smoothed surface plot with external color schemes based on cluster or categorical attribute.
An example is given in Figure Here the scatterplot for the Swiss roll dataset given in Figure 5 is plotted with a loess surface. The loess surface helps users explore overall patterns in performance. For example in the Smacof vs. PCA comparison, Smacof has stronger agreement towards the center of the roll, but worse agreement towards the outside of the roll. The area plots implemented in QVisVis can be utilized for comparing single dimensionality reduction techniques with random agreement or for comparing dimensionality reduction techniques against one another.
Per- formance lift area plots for six different dimensionality reduction techniques vs. On each diagram, each of the lift techniques is assigned a color.
If two techniques overlap then a color intermediate to the two colors is shown. One can see that across all k, PCA outperforms most of the techniques, except for Smacof, which maintains a slight, but consistent outperformance.
A few of the diagrams are particularly interesting. SmacofLOC has a slight area of outperformance for very small k, but then is dominated by PCA, which again emphasizes the local nature of the algorithm. A range of multidimensional scaling based methods have been utilized in marketing mapping applications [25, 64].
To demonstrate the use of QVisVis on real-world data, several algorithms were tested on a set of consumer data2 , which gives music, movie, and lifestyle preferences for a young adults.
The dataset has items and questions, each requiring a Likert scale response from 1. It has a few missing values, which account for 0. The t-SNE method [46, 47], which is a variant of stochastic neighborhood embedding [29], was implemented, as it is optimized to give good local neighborhood recovery. It has been used in marketing mapping applications, e. Distance MDS implemented with Smacof and LLE were selected as benchmarks, as they are designed for global and local recovery respectively.
To create an overall comparison, performance lift diagrams were created to compare recovery across all k. For Smacof, basic MDS was implemented with an ordinal distance transformation. The resulting visualizations are shown in Figure Isomap on Sphere. This performance effect is present for all three perplexity values, though it decreases as p increases. As could be expected, as k increases, the heatmaps go from mostly blue t-SNE outperformance to red Smacof outperformance.
The aggregate aggrement statistics emphasize this pattern. The heatmaps are given in Figure Smacof on Large Diameter Regular Torus. For the three local plots, the t-SNE solution was used and for the three global plots, the Smacof solution was used. As expected, t-SNE strongly dominates in the local plots and Smacof outperforms in the global plots. There also seems to be a slight effect, where Smacof does relatively better towards the center of the configuration and t-SNE does better towards the edge of the configuration.
Overall, in a situation where local neighborhood agreement is important, for example, where a marketing manager who wishes to examine relationships betweeen individual customers, then the results show that t-SNE has the best recovery. However, if global agreement is important, for example, if a marketing manager wishes to examine segments across an entire customer dataset, ordinal MDS implemented with Smacof gives very good performance though only slightly better than t-SNE with the maximum value of p.
In the context of calculating agreement, this example showed how multiple visualizations can give different insights into comparative performance, across both neighborhood size and the geography of the solution configuration. This review is used to build a framework for analyzing and visualizing dimensionality reduction performance entitled QVisVis. It is a practical framework, designed to help evaluate dimensionality reduction techniques and to give insight into the performance of the techniques beyond a simple numeric quality metric.
The overall framework allows for di- mensionality reduction algorithm performance to be analyzed versus random agreement and against other techniques. It allows users to examine the local vs. Much further work can be done on the framework and software. The current visualizations show solution agree- ment, but other relevant information, such as measures of item centrality of the point could be shown [50]. The software has the ability to draw visualizations across different technique parameter settings, but this ability was not emphasized in this paper and future work could help develop visual based methods of parameter tuning.
The visualizations are currently static. Animated visualizations could parsimoniously help users see how visualization performance changes over time and a wide variety of neighborhood sizes and parameter settings. Finally, a wider range of quality measures could be implemented. For example. However, in practical vi- sualization contexts, human perception of visualization is important. There have been several article on incorporating human perceptions of visualizations into evaluation of visual embeddings, for example, in [79], a user survey is used to analyze human perceptions of visualizations with respect to quantitative quality metrics.
In a similar fashion, other work has gathered survey data on the quality of cluster seperation for different types of scatterplot [69] and on testing how well visualizations allow users to carry out tasks, such as analyzing the number of clusters and relative distances in visualizations [21].
The combination of quantitative quality metrics and the analysis of human interaction with visualizations and could lead to further insights into visualization quality. The scatterplots created in the QVisVis framework can be used to show i the relative performance of different techniques, ii performance vs. For parsimony, the examples in this paper are quite simple. However, by combining i, ii, and iii, we could get many hundreds of scatterplots.
These could be displayed in scatterplot matrices SPLOMs , but this is not practical when there are large number of scatterplots. These characteristics can then be used to explore and cluster scatterplots [19] or mathematical transforms of scatterplots [84]. Combining agreement metrics with overall scatterplot scagnostics could give a deeper understanding of the features and quality of dimensionality reduction algorithms.
Other Techniques on Swiss Roll. Akkucuk, U. Isomap: A comparison of two nonlinear mapping algorithms. Journal of Classification 23 2 , — 3. Andrew, V. Information Visualization 10 4 , — 4.
Aupetit, M. Neurocomputing 70 7 , — 5. Belkin, M. Neural Computation 15 6 , — 6. Bertini, E. Borg, I. Springer, New York, NY 8. Bradley, A. Pattern Recognition 30 7 , — 9. Buja, A.
Journal of Computational and Graphical Statistics 17 2 , — Carpendale, S. Springer, Heidelberg, Germany Carroll, J. Chen, L. Journal of the American Statistical Association , — Journal of Machine Learning Research 14, — Cleveland, W.
Journal of the American Statistical Association 74 , — Coifman, R. Applied and Computational Harmonic Analysis 21 1 , 5—30 Coimbra, D.
Informa- tion Visualization 15 2 , — Cutura, R. Dang, T. DOI Dijkstra, E. Numerische Mathematik 1 1 , — Etemadpour, R. France, S. Algorithms from and for Nature and Life. Springer Verlag, Heidelberg, Germany In: P. Perner ed. Expert Systems with Applications , — Groenen, P. Journal of Classi- fication 12 1 , 3—19 Hand, D. Machine Learning 45 2 , — Hanley, J. Radiology 1 , 29—36 Hinton, G.
In: S. Becker, S. Thrun, K. Obermayer eds. Hubert, L. Journal of Classification 2 1 , — Ingram, S. In: A. MacEachren, S. Miksch eds. Jankun-Kelly, T. Karatzoglou, A. Journal of Statistical Software 11 9 , 1—20 Kaski, S. BMC Bioinformatics 4 1 , 48 Kraemer, G. Kruskal, J. Psychometrika 29 2 , — Laskowski, P.
The American Cartographer 16 2 , — Lee, J. In: Y. Saeys, H. Maximising opportunity to learn. Maximising opportunity to learn - chapter 3. Activating hard thinking. Activating hard thinking - chapter 4. Review methodology, overview of studies reviewed, references and further reading.
Review methodology- chapter 5. The webinars gave Cambridge International teachers more insights into the findings of the Review and tips on how they can integrate the findings into their professional development. Watch the Great Teaching Toolkit webinar. The presentation slides are available and Cambridge Assessment published a blog on the toolkit.
Cambridge principles into practice - languages guide for schools. You do not need to install any other software. You can also install PDF24 as an app on your smartphone. Then click on the "Install" icon in the upper right corner of the address bar or add PDF24 to your start screen via the Chrome menu. Yes, Windows users can also use PDF24 offline, i. Simply download the free PDF24 Creator and install the software. Users of other operating systems should continue to use PDF24 Tools.
0コメント