Mohammad Jeelani, Michael Prieto (2025). Using the Analytic Hierarchy Process (AHP) to Prioritize AR Device Considerations. In: Human-Computer Interaction & Emerging Technologies. AHFE Open Access, vol. 195, pp. 305–313. https://doi.org/10.54941/ahfe1006249
When designing and developing emerging technologies, understanding user priorities for different product considerations is essential for ensuring that products meet user needs and expectations, leading to better adoption, satisfaction, and overall success in the market. Understanding and characterizing user priorities is critical for making design trade-offs and aligning design and development teams to high-value efforts. This study explores the application of the Analytic Hierarchy Process (AHP) to determine how users prioritize key product considerations in the selection of Augmented Reality (AR) devices. While other prioritization techniques such as Maximum Difference Scaling (MaxDiff) and the ranking method are commonly used to better understand user priorities, AHP provides a more accurate and reliable approach, resulting in priority weights that reflect both the relative importance and the intensity of user preferences across multiple factors.For this study, AR device users completed a web-based survey that included a series of pairwise comparison prompts to prioritize six AR device considerations: comfort, features and functionality, lens quality, price, battery life, and aesthetics. For each of the fifteen pairwise comparisons, participants first indicated which device consideration was more important to them. Then, the participants rated the strength of their preference for the more important factor. The results from a sample of 37 participants revealed comfort was the most significant factor, followed by features and functionality, and lens quality. Price and battery life were also important but ranked lower, while aesthetics was deemed the least important consideration.Utilizing AHP with a panel of remote participants proved to be an effective human-centered approach for prioritizing device considerations for AR devices. The outputs from the AHP analysis not only establish a priority but also priority weights which offer deeper insights into exactly how important each factor is, revealing the relative intensity of user preferences. These priority weights can also be used to quantitatively evaluate products and prototypes, providing a more objective basis for comparison and decision-making. The methods utilized in this study facilitate a deeper understanding of user preferences and priorities which can be applied to the development of many products and emerging technologies.
The successful design and development of emerging technologies hinges heavily on delivering products that meet user needs and expectations. If priorities are misaligned during new product development, it is likely that market adoption and user satisfaction will suffer. Understanding design trade- offs is critical for aligning development efforts with high-value priorities. By balancing competing demands effectively, product development teams can increase the likelihood of creating successful technologies that deliver long-term value.
When selecting an augmented reality (AR) device, potential users take several factors into consideration. For many users of AR technology, a bulky or unappealing form factor is a deal-breaker as comfort and aesthetics play a crucial role in perceptions for social acceptability and the overall user experience. Technical capabilities such as the features and functionality of the device as well as lens quality are also critical for the overall utility of the product, but may introduce constraints to the form factor, battery life, and price of the product. Understanding how users prioritize these trade-offs is essential for delivering a product that consumers will purchase and use.
This paper details a study on the application of the analytic hierarchy process (AHP) for prioritizing AR device considerations. By leveraging AHP, this research aims to provide insights into how different product considerations are prioritized by users, offering a structured methodology for understanding user preferences. The findings from this study can be used by product development teams in the AR space to guide design trade-offs, ensuring alignment between user preferences and product design.
METHODS FOR REVEALING USER PRIORITIES Several multi-criteria decision-making (MCDM) techniques can be employed to reveal user priorities, each with advantages and disadvantages. Three commonly used techniques for uncovering user priorities include the ranking method, Maximum Difference Scaling (MaxDiff), and the Analytical Hierarchy Process (AHP). These methods all help to guide product development efforts by identifying which product considerations users prioritize the most.
The ranking method is perhaps the most simplistic and easily executable method for prioritizing product considerations. For the ranking method, criteria are ranked in order of importance. Weights are then calculated using one of three methods: rank sum, rank exponent, and rank reciprocal (Odu, 2019). The rankings from multiple respondents are aggregated by calculating the geometric mean for each criterion across all participants MaxDiff is another commonly used method for measuring preferences for multiple criteria. “In MaxDiff, respondents answer which of the several presented items is the best and which is the worst, so the remaining items are in the interval between these margins” (Lipovetsky and Conklin, 2015). For each criterion, the number of “best” responses and the number of “worst” responses are tallied for all respondents. A utility score is then calculated for each criterion by subtracting the number of “worst” responses from
Using the Analytic Hierarchy Process (AHP) to Prioritize AR Device Considerations 307
the number of “best” responses and the full set of criteria are prioritized accordingly.
The Analytic Hierarchy Process (AHP) is a method “used to derive ratio scales from both discrete and continuous paired comparisons” (Saaty, 1987). AHP requires respondents to make a series of pairwise comparisons on a bipolar ratio scale of 1 to 9, where the values are reciprocal, meaning if the more important criteria is rated as X, the less important criteria is rated as 1/X. The AHP rating scale is shown in Table 1. The responses are then organized into a square matrix where the values are normalized and used to calculate priority weights for each criterion. AHP also includes a consistency check to ensure that respondents’ judgments are coherent. AHP can also be used to evaluate alternatives against these weighted criteria.
Table 1: AHP rating scale.
Table available in the original PDF paper.
Importance Numerical Rating Recipricol Extreme 9 1/9 Very Strong to Extreme 8 1/8 Very Strong 7 1/7 Strong to Very Strong 6 1/6 Strong 5 1/5 Moderate to Strong 4 1/4 Moderate 3 1/3 Equal to Moderate 2 1/2 Equal 1 1
While the ranking method, MaxDiff, and AHP are all effective methods for revealing user priorities, they each have their differences. The ranking method is the easiest method to execute, however, it does not capture the nuanced differences in the relative importance of criteria as effectively as AHP and MaxDiff. Out of these three methods, AHP is the only one that requires respondents to compare criteria directly against one another. While MaxDiff also captures relative preferences by focusing on extreme choices (best and worst) and then derives priorities by calculating utility scores, AHP uses pairwise comparisons between individual criteria, leading to more accurate and precise weights that reflect the relative importance of each criterion. AHP is also the only method of these three that includes a consistency check and a direct method for evaluating alternatives against the weighted criteria. Table 2 illustrates the differences between these three methods.
Table 2: Comparison of MCDM methods.
Table available in the original PDF paper.
Criteria AHP MaxDiff Ranking Method Ranking in order of preference
Choice-based data (best and worst for several subsets)
Input data Pairwise comparisons for every combination of criteria
Continued
308 Jeelani and Prieto
Table 2: Continued Criteria AHP MaxDiff Ranking Method
Table available in the original PDF paper.
Criteria scores and rank order
Outputs Ratio scale weights Interval scale utility scores and derived relative weights
Low - rank based algorithms
Data analysis complexity
High - matrix creation, consistency checks, eigenvector calculation
Moderate - Calculate utility scores and relative weights Assumed No
Consistency ratio calculated
Consistency check
Low - Relative rankings only, does not reflect magnitude of preference
Insight depth High - Direct ratio scale weights reflect relative importance based on absolute importance judgments
Moderate - Normalized utility scores reflect preference strength based on observation choices None Indirect methods using relative weights
Method for evaluating alternatives
Pairwise comparisons between alternatives against weighted criteria
USING AHP When using AHP, a hierarchy is first created for the decision problem as shown in Figure 1. The top level of the hierarchy is the goal or decision to be made. The next level consists of the criteria that affect the decision. For complex decision problems, additional lower levels can be created for subcriteria. The lowest level of the hierarchy consists of alternatives that will be evaluated against the criteria.
During the AHP process, decision-makers perform pairwise comparisons for every combination of criteria. The traditional method for soliciting this feedback from respondents is to have them complete a matrix as shown in Table 3. In this example, the respondent indicated a moderate preference for bananas over apples, a very strong preference for bananas over oranges, and a strong preference for apples over oranges. Once this matrix is complete, it is relatively easy to transfer it to a decision-making software or spreadsheet for calculations.
Table 3: Example pairwise comparison matrix.
Table available in the original PDF paper.
Banana Apple Orange Banana 1 3 7 Apple 8 1/8 5 Orange 7 1/7 1
Using the Analytic Hierarchy Process (AHP) to Prioritize AR Device Considerations 309
Figure 1: AHP hierarchy.
Since completing the matrix requires respondents to have a general understanding of AHP and how the matrix works, using a bipolar rating scale is often more efficient and intuitive. This allows respondents to focus on the relative importance between two criteria at a time rather than requiring an understanding of the entire matrix at once. The disadvantage to using these rating scales rather than respondents directly completing the matrix is that the responses from the rating scales will need to be transferred to the matrix, adding an additional step for analysis. The three rating scales that correspond with the ratings in the matrix for the previous example are shown in Figure 2.
Figure 2: AHP rating scales for example decision problem.
310 Jeelani and Prieto
The values in the matrix are normalized by dividing each value by the sum of the values in its column. Weights for each criterion are calculated by averaging the values in each row. A consistency check is then performed to ensure that the pairwise comparisons are logically consistent by calculating a consistency ratio (CR) for each pairwise comparison matrix. The priority weights and consistency ratio for the previous example are shown in Table 4. When performing AHP for multiple respondents, the weights of individual respondents are aggregated by calculating the geometric mean of the criteria weights across respondents.
Table 4: Example pairwise comparison matrix.
Table available in the original PDF paper.
AHP Consistency Check Banana 0.643 64.3% Consistency OK 8% Apple 0.283 28.3% Orange 0.074 7.4%
METHODOLOGY
A web-based survey was utilized to solicit insights from 37 current users of lightweight AR devices. Participants were recruited by Userlytics, an online user testing platform. Participants were screened for owning select lightweight AR devices: RayBan Meta, TCL Rayneo, INMO Air, and Google Glass. The Samsung VIVO Air, Apple Vision Pro, and Meta Quest 3 were also provided as selectable options in the screener, however, individuals selecting any of these options were rejected. In addition to AR device ownership, participants were also asked about their gender, the type of community they lived in (city/urban, suburban, or rural), household income, and employment status. Participants who passed the screener were provided with a link to a survey hosted by Qualtrics.
Participants completed a total of 15 pairwise comparisons for six AR device considerations: comfort, features and functionality, lens quality, price, battery life, and aesthetics. The 15 pairwise comparisons included every combination of the 6 AR device considerations. The hierarchy used for this study is shown in Figure 3.
Rather than having respondents complete the traditional AHP matrix or pairwise rating scales, a staged approach was used leveraging the scripting features available in Qualtrics. For each pairwise comparison, participants were first presented with three buttons where they could select the criteria they valued more as well as an “equal” button if they valued both criteria equally. This prompt is shown in Figure 4. If participants indicated that one consideration was more important than the other, a slider was then displayed where they could select their level of preference for that criteria over the other as shown in Figure 5.
Using the Analytic Hierarchy Process (AHP) to Prioritize AR Device Considerations 311
Figure 3: AHP hierarchy for lightweight AR device considerations.Figure 4: Criteria selection prompt.Figure 5: Importance slider for prioritized criteria.
DEMOGRAPHICS The sample for this study was significantly more male, with males representing 63.3% of the sample. Participant ages ranged from 20 to 55 years old, with a median age of 35.7. Participant income levels ranged from $25,000 to $174,999, with higher representation in the $100,000– $124,999 income bracket. 73% of participants were users of the Rayban Meta glasses, while 24% used the Google Glass and 3% used the INMO Air. The majority of participants indicated that they owned their AR device for a year or less.
ANALYSIS AND RESULTS The survey data was exported as a comma-separated value (CSV) file from Qualtrics. A Python script was then used to translate the CSV data for the pairwise comparisons to matrices for each respondent. Once all of the matrices were created, Excel was used to calculate priority weights and consistency ratios. Matrices for 9 of the 37 respondents were eliminated due to a CR of more than 10%.
312 Jeelani and Prieto
The results from this study indicate that comfort is the most important AR device consideration, followed closely by features and functionality, and lens quality. Aesthetics emerged as the least important AR device consideration. The AHP results for this study are shown in Table 5.
While several MCDM methods are available for understanding user priorities, AHP is a preferred method due to its ability to provide in-depth insights by capturing the importance of various criteria through pairwise comparisons. The use of a consistency check further ensures the reliability of the results, verifying that respondents’ judgments are logically coherent. The approach of using interactive staged prompts for criteria selection and importance value sliders in Qualtrics further enhanced the ease of use and intuitiveness for data collection.
This study revealed that comfort is the most important AR device consideration, followed by features and functionality, and lens quality with priority weights of 20%, 19%, and 19% respectively. Aesthetics was found to be the least important consideration. These insights would suggest that design teams focus on delivering a comfortable product, specifically optimizing fit and minimizing weight, even if it results in a more expensive product.
Future research opportunities include refining the hierarchy to better represent device considerations that AR users consider when making a purchasing decision. Considering the majority of respondents owned Rayban Meta glasses, which follow a widely acceptable form factor, adding social acceptability attributes may provide a broader spectrum for understanding user needs. Including additional levels of subcriteria would also allow for deeper insights. Increasing the sample size will increase confidence in the results and will capture a wider range of perspectives. Repeating the study with specific user archetype groups, such as travelers, fitness enthusiasts, and working professionals, is also likely to reveal nuances for user priorities between these groups. Additional opportunities include using these weights and pairwise comparisons to evaluate commercially available and prototype devices against these criteria.
REFERENCES
Lipovetsky, Stan, and Michael Conklin. “MaxDiff Priority Estimations with and without HB-MNL.” Advances in Adaptive Data Analysis, vol. 07, no. 01n02, 1 Apr. 2015, http://doi.org.10.1142.s1793536915500028.
Using the Analytic Hierarchy Process (AHP) to Prioritize AR Device Considerations 313
Odu, G. O. “Weighting Methods for Multi-Criteria decision-making Technique.”
Journal of Applied Sciences and Environmental Management, vol. 23, no. 8, 11 Sept. 2019, pp. 1449–1457, https://doi.org/10.4314/jasem.v23i8.7. Saaty, R. W. “The Analytic Hierarchy Process – What It IS and How It Is Used.”
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-71-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional