The following frequently asked questions (FAQs) pertain to the QualityMetric Health Outcomes™
Scoring Software 2.0 and have been categorized below to support users of this software in resolving
common questions or issues.
General
Science/Research
Technical
I am using an older version Health Outcomes™ Scoring Software. Where can I find
help and support for previous versions?
Help and Support for previous versions of the Health Outcomes™ Scoring Software may be found
here.
How does the latest version of the QualityMetric Health Outcomes™ Scoring
Software 2.0 differ from the previous version?
While maintaining the same features as the previous version, the latest version of the QualityMetric Health
Outcomes™ Scoring Software 2.0 incorporates many improvements and enhancements. These are reflected
in the following comparison table:
|
Feature
|
Original Version
|
Version 2.0
|
|
Reports
|
DQE/MDE/summary
|
Individual patient, individual provider, total sample, sample by gender, sample by age, DQE/MDE/ summary, tabular
(not all reports are available for all surveys).
|
|
Data entry options
|
.CSV file
|
On-screen administration, Data Entry Grid, .CSV file
|
|
Missing Data Estimation, Data Quality Evaluation
|
Optional
|
Standard
|
|
Storage of data by project/survey
|
N/A
|
Standard
|
|
Administration and scoring of SF-10™ Health Survey for Children
|
N/A
|
Standard
|
|
Medical expenditure risk prediction
|
N/A
|
Standard with individual provider and total sample reports for both versions of patient
SF-36® Health Survey and SF-12® Health Survey
|
|
First stage depression screening
|
N/A
|
Standard with total sample and gender aggregate reports
|
|
Comparison of percent of sample below/at/above population norm
|
N/A
|
Standard with total sample and gender aggregate reports
|
|
Tracking of results over multiple administrations
|
N/A
|
Standard with individual patient and individual provider reports
|
|
Multi-user usage mode
|
N/A
|
Standard
|
Are all of the QualityMetric Health Outcomes™ Scoring Software 2.0
reports available for all of the QualityMetric™ Health Surveys?
Not all reports are available for all of the QualityMetric™ Health Surveys.
The reports that can be generated for each of the surveys are reflected in the following table:
|
Reports
|
SF-36v2™ Health Survey
|
SF-36® Health Survey
|
SF-12v2™ Health Survey
|
SF-12® Health Survey
|
SF-8™ Health Survey
|
SF-10™ Health Survey for Children
|
| Individual |
 |
|
|
|
|
|
| Provider |
|
|
|
|
|
|
| Total sample |
|
|
|
|
|
|
| Age |
|
|
|
|
|
|
| Gender |
|
|
|
|
|
|
| DQE/MDE/summary |
|
|
|
|
|
|
| Tabular |
|
|
|
|
|
|
Which of the QualityMetric™ Health Surveys can be computer administered and
scored via QualityMetric Health Outcomes™ Scoring Software 2.0?
The SF-36® Health Survey, SF-36v2™ Health Survey, SF-12® Health Survey, SF-12v2™
Health Survey, SF-8™ Health Survey, and SF-10™ Health Survey for Children can be administered
via the software. However, administration of any of the QualityMetric™ Health Surveys requires a
separate license.
Who should use the QualityMetric Health Outcomes™ Scoring Software 2.0?
Anyone with raw data collected through administration of one of the QualityMetric™ Health Surveys.
Also, users can now use the QualityMetric Health Outcomes™ Scoring Software 2.0 to have responses
to any of the QualityMetric™ Health Surveys entered directly via computer-based administration of
the survey.
Why should I buy the QualityMetric Health Outcomes™ Scoring Software 2.0 when the scoring
algorithms are published in your manuals?
The QualityMetric Health Outcomes™ Scoring Software 2.0 is the only software recommended by the developers
of the SF-36® Health Surveys and other QualityMetric™ Health Surveys. The software saves the time it
takes to write the code necessary to score questionnaire items, health domain scales, and component summary
measures. It minimizes the chance of making scoring errors and adds noteworthy features that are not otherwise
available together. These features include:
-
Data quality evaluation
-
Detailed instruction for data formatting, step-by-step integrated help, and a User's Guide
-
Options for handling missing data to minimize bias and subject loss
-
First stage depression screening
-
Predicted average monthly medical expenditure risk
What does the QualityMetric Health Outcomes™ Scoring Software 2.0 do?
The QualityMetric Health Outcomes™ Scoring Software 2.0 is now available for use in scoring and reporting individual-patient
or group-level results for the SF-36® Health Survey, SF-36v2™ Health Survey, SF-12® Health Survey, SF-12v2™
Health Survey, SF-8™ Health Survey, and SF-10™ Health Survey for Children. It includes Data Quality Evaluation (DQE),
one of the most important procedures to identifying problems with data quality, which is key to achieving precise and valid health
status measurement. It also includes the Missing Data Estimation (MDE), providing options for recovering cases with missing data, which dramatically heightens the value of collected data for any type of population monitoring effort. It also provides first stage depression screening and predicted average monthly medical expenditure risk.
Scoring of the Utility Index is available as an optional component of the QualityMetric Health Outcomes™ Scoring Software 2.0.
Vital for health economics, the Utility Index ascertains the magnitude of measured change in health status for health economics (e.g. cost-effectiveness,
quality adjusted life years).
With the QualityMetric Health Outcomes™ Scoring Software 2.0, the user can computer-administer each of the QualityMetric™ Health Surveys;
however, a separate license is required for administration of the surveys.
What is included with the QualityMetric Health Outcomes™ Scoring Software 2.0?
-
An integrated, step-by-step help feature in the software
-
A customer-focused, easy to understand User's Guide
-
Software offered via download
-
An Activation Key sent to you via email
What is not included?
A survey license is not included with the software. All users of the QualityMetric Health Outcomes™ Scoring Software 2.0 should have an up-to-date license to use any one or more of the QualityMetric™ Health Surveys.
Is the QualityMetric Health Outcomes™ Scoring Software 2.0 HIPAA compliant?
The QualityMetric Health Outcomes™ Scoring Software 2.0 is a tool that an organization can use to score HRQOL data and feel confident that the data was
scored correctly. The QualityMetric Health Outcomes™ Scoring Software 2.0 itself is either compliant with HIPAA or not depending upon the activities of the
user, not the software. HIPAA compliance depends upon the practices of the individual using the software. If an organization is HIPAA compliant and you use
the QualityMetric Health Outcomes™ Scoring Software 2.0 appropriately, then it should comply with HIPAA guidelines. Entry of personally identifiable health-related
data into the software, which may affect HIPAA compliance, is an optional feature for the user.
Could you explain how your licensing works? Annual fee? Restriction on number of users?
Licensing of the QualityMetric Health Outcomes™ Scoring Software 2.0 is for one year. The initial purchase fee covers the first year license. Following the
first year, there are annual license fees for the QualityMetric Health Outcomes™ Scoring Software 2.0. An optional annual support contract may be purchased.
The licensing agreement starts on the purchase date. The license to use the QualityMetric Health Outcomes™ Scoring Software 2.0 is a per-seat license, that is,
each user of the software must have a license to do so. The QualityMetric Health Outcomes™ Scoring Software 2.0 has pricing for both single and multiple users at a site.
Please call QualityMetric for volume licensing information at (800) 572-9394.
I am interested in using your product in a clinical setting with medical patients for purposes of individual care planning as well as developing clinic services. Will the QualityMetric Health Outcomes™
Scoring Software allow me to create reports for individual patients?
The QualityMetric Health Outcomes™ Scoring Software allows the creation of individual patient reports for all of the QualityMetric™ Health Surveys.
Does the FDA accept the algorithms used by the QualityMetric Health Outcomes™ Scoring Software 2.0 for Data Quality Evaluation and Missing Data Estimation?
The FDA has consistently accepted the scoring algorithms recommended by the developers of the SF-36® Health Surveys and other QualityMetric™
Health Surveys as programmed in the QualityMetric Health Outcomes™ Scoring Software 2.0, including those applied to Data Quality Evaluation (DQE) and Missing Data Estimation (MDE).
DQE makes it easy to compare the results from psychometric tests (e.g. scoring assumptions, reliability) in a clinical trial in relation to accepted standards in the field. MDE algorithms minimize
subject loss at baseline and follow-up and minimize bias in score estimates.
What is the function of the Missing Data Estimator (MDE)?
MDE is a feature of the QualityMetric Health Outcomes™ Scoring Software 2.0 that vastly improves methods of data recovery, making it possible to calculate health domain scale and component
summary measures scores for respondents who do not answer every survey item and for whom scores would be missing using standard scoring algorithms. These improvements result in a substantial
reduction in subject loss and a minimization of bias in score estimates.
For more information on the development and validation of the MDE algorithms, see:
Kosinski M, Bayliss MM, Bjorner JB, Ware JE, Jr. Improving Estimates of SF-36® Health Survey Scores for Respondents With Missing Data. Medical Outcomes Trust Monitor, 2000; 5(1):8-10.
What is the Utility Index?
The Utility Index is an optional, value-added feature for the QualityMetric Health Outcomes™ Scoring Software 2.0. This feature offers scoring based on the method developed by Brazier
and his colleagues. Although the SF-36® Health Survey was not originally designed for use in economic evaluation, research has shown that a
meaningful health state classification, called the Utility Index, can be created by applying a scoring method that focuses on seven domains covered by the
SF-36® Health Survey. These domains are:
-
Physical Functioning
-
Role-Physical
-
Social Functioning
-
Bodily Pain
-
Role-Emotional
-
Mental Health
-
Vitality
The resulting Utility Index can be used to assess the cost-effectiveness of various health care interventions and
"quality adjusted life years" (QALYs). At present, the Utility Index can only be calculated from
SF-36® Health Survey, SF-36v2™ Health Survey,
SF-12® Health Survey and SF-126v2™ Health Survey data.
Brazier J, Usherwood T, Harper R, Thomas K. Deriving a performance-based single index from the UK SF-36 Health Survey. Journal of Clinical Epidemiology. 1998;51(11):1115-1128.
Brazier J, Roberts J, Deverill M. The estimation of preference-based measure of health from the SF-36. Journal of Health Economics, 2002; 21:271-292.
What is the first stage depression screening?
The first stage depression screening is a preliminary assessment of a respondent's risk for screening positive for depression. A positive score is defined as a Mental Component Summary (MCS)
measure score less than or equal to 42. A comparison of the percentage of respondents in a sample identified as being at risk for screening positive for depression is compared with that for
the general population in the QualityMetric Health Outcomes™ Scoring Software 2.0 total sample and gender aggregate reports.
What is the medical expenditure risk level?
The medical expenditure risk level is the predicted average monthly medical expenditure risk for respondents. Medical expenditure is predicted through a regression-based equation, which weighs
a respondent's health-related quality of life data, gender, and age. In addition to the predicted expenditure, the QualityMetric Health Outcomes™ Scoring Software 2.0 provider report and
aggregate report also indicate the percentage of the general population that is surpassed by the sample's average predicted medical expenditure risk.
What is the difference between 0-100 scoring and norm-based scoring (NBS)?
Scoring on the 0-100 metric provides a score that represents the percentage of the total possible score that the survey respondent received. This score is computed by: 1)
subtracting the lowest possible raw score from the respondent's actual raw score; 2) dividing this number by the possible raw score range; and 3) multiplying this number by 100.
For example, on the SF-36v2™ Health Survey, the lowest possible raw score on the Physical Functioning scale is 10 and the highest possible raw score is 30. Therefore, the
possible raw score range is 20. If a respondent received a raw score of 21, then it would be transformed to the 0-100 metric as follows:
[ (21-10) ] x 100 / [ 20 ] = 55
where the lowest possible raw score = 10 and the possible raw score range = 20.
Norm-based scoring (NBS) provides a score that represents where a person scores in relation to a known population. The interpretation of a score is more meaningful when compared
to that of a reference group than when viewed in isolation. For example, if you received a score of 75 out of 100 on a math test, you would probably want to know how your score
compared to the scores of others who took the same math test before you concluded that this was a poor score or a good score. If the class average was 80, then your score would be
below average and you might be disappointed with your score; if the class average was 70, then your score would be above average and you might be pleased with your score. NBS algorithms
transform all adult QualityMetric™ Health Survey scores to a common metric where the mean is 50 and the standard deviation is 10 in the U.S. general population. The norm-based score is
computed by: 1) calculating a z-score (a standardized score that tells you how far a score varies from the known mean in standard deviation units), and 2) transforming this score to have a
mean of 50 and a standard deviation of 10.
Step 1: To calculate a participant's z-score on the Mental Health scale, subtract the known 0-100 population mean on the Mental Health scale from the participant's 0-100 score on the Mental
Health scale and divide this number by the known population standard deviation.
MH_Z= (MH - 74.98685) / 17.75604
where MH = the participant's 0-100 score For example, if a participant had a score of 74.98685 on the Mental Health scale, then the participant's Mental Health z-score would be equal to 0.
MH_Z = (74.98685 - 74.98685) / 17.75604
MH_Z = (0) / 17.75604
MH_Z = 0
Step 2: To calculate a norm-based score with a mean of 50 and a standard deviation of 10 in the U.S. general population, simply multiply the Mental Health z-score by 10 and then add 50. Thus,
Norm-Based MH = 50 + (MH_Z * 10)
Norm-Based MH = 50 + (0 * 10)
Norm-Based MH = 50
where MH_Z = the participant's z-score.
Thus, if a respondent received a score of 55 on one of the health domain scales using the 0-100 scoring metric, this would indicate that the respondent received 55% of the total possible score on that scale. If a respondent received a score of 55
on one of the health domain scales using the NBS metric, this would indicate that the respondent scored a half a standard deviation (5 NBS points) above the population mean (50) on that scale.
Are scores comparable across the QualityMetric™ Health Surveys?
Scores using the 0-100 metric are not directly comparable across the SF-36® Health Survey, SF-36v2™ Health Survey, SF-12® Health Survey, SF-12v2™ Health Survey, or the SF-8™ Health Survey due
to differences in the numbers of items and response options. Norm-based scoring (NBS) standardizes the 0-100 metrics for all forms to have the same mean (50) and standard deviation (10) in the U.S. general population.
As a result, NBS achieves directly comparable results across the QualityMetric™ Health Surveys developed for adults.
If I want to compare the norm-based scores I got from my SF-36v2™ Health Survey data to 0-100 scores that were published in a study that used the SF-36® Health Survey, can the QualityMetric Health Outcomes™
Scoring Software 2.0 help me do that?
Yes. The QualityMetric Health Outcomes™ Scoring Software 2.0 allows users of the SF-36® Health Survey or SF-36v2™ Health Survey to make "backward" or "forward" comparisons of their scores to scores obtained on the alternate
version of the measure and/or scores presented in a different metric. This feature makes it possible for users to make direct comparisons of scores across data sets that utilize different versions of the SF-36® Health Survey and to
published norms of data obtained on either form.
For example, if a user has recently adopted the SF-36v2™ Health Survey into a study but would like to compare the data obtained on this form to 0-100 data she had already collected in the past with the original SF-36® Health Survey,
the QualityMetric Health Outcomes™ Scoring Software 2.0 can provide "backward" conversion and provide an estimate of the health domain scale scores on the SF-36® Health Survey from the actual data obtained on the SF-36v2™ Health Survey.
Alternatively, a user might want to make a "forward" conversion of their SF-36® Health Survey data scored on the 0-100 metric to take advantage of the current norm-based scoring (NBS) method.
The options for score conversion are available for the SF-36® Health Survey and SF-36v2™ Health Survey using the NBS calculator that is included in the software.
What is the difference between convergent validity and discriminant validity?
Convergent validity and discriminant validity are both forms of construct validity. Construct validity is defined as the extent to which a measure correlates with other measures or "constructs" in a manner consistent with theory. Convergent validity
is an aspect of construct validity in which two methods of measuring similar constructs are shown to be associated. Tests of convergent validity are done to determine whether, consistent with theory, constructs that should be related to each other
are observed to be related to each other. For example, items from the QualityMetric™ Health Surveys that measure the construct of "Physical Functioning" should be related to the Physical Component Summary (PCS) measure score, while items that
measure the construct of "Mental Health" should be related to the Mental Component Summary (MCS) measure score.
Discriminant validity is an aspect of construct validity in which a measure is shown to correlate higher with concepts it is intended to measure than with concepts it is not intended to measure. Tests of discriminant validity are conducted to determine
whether, consistent with theory, measures of constructs that should not be strongly related to each other are not observed to be strongly related to each other. For example, if the Physical Functioning items on the QualityMetric™ Health Surveys are
designed to measure a construct of "Physical Health" and not a construct of "Mental Health," then the relationship between the Physical Functioning items and the PCS score should be stronger than the relationship between the Physical Functioning items and
the MCS score.
Tests of convergent and discriminant validity are important indicators of data quality. If the data does not pass the test, check your data for coding and data entry errors.
Does the QualityMetric Health Outcomes™ Scoring Software 2.0 have the capacity to incorporate data collected using the Health Status Questionnaire (HSQ),
which differs from the SF-36® Health Survey by increasing the responses on one of the SF-36® Health Survey scales?
No. The QualityMetric Health Outcomes™ Scoring Software 2.0 will score QualityMetric™ Health Surveys only. If you try to score a modified version of an SF-36® Health
Survey that contains additional response options, the software will consider responses on that additional option "out of range" and will replace them with a missing value.
How valid are the 1998 norms given that we are now in the year 2007?
QualityMetric Incorporated acknowledges the need to update its norms periodically. The original norms were based on data obtained in 1990. The 1998 norms (default in the QualityMetric Health
Outcomes™ Scoring Software 2.0) do not vary significantly from the 1990 norms, but there are small differences between the two sets of norms. Based on our past experience, QualityMetric does
not expect to find significant differences between our 1998 norms and norms that are updated.
On the summary report, there are the transformed scores and the norm-based scores. Do you have a report that would show the z-scores?
We do not provide z-scores in the summary report. However, the norm-based scores that we do provide are T-scores, which are simply transformations of z-scores:
Norm-Based Score = (z-score * 10) + 50
Therefore, you could use the following formula to get z-scores:
z-score = (Norm-Based Score - 50)/10
For example, if you had a Physical Functioning norm-based score (PF_NBS) of 55, then the z-score for Physical Functioning (PF_Z) would be computed as:
PF_Z = (PF_NBS -50)/10
PF_Z = (55 - 50)/10
PF_Z = 5/10
PF_Z = .5
How do I get the references you mentioned?
You can get a list of relevant references by clicking on the following link: references.
Can you repeatedly have a single patient complete the SF-36v2™ Health Survey over a 4-month period? If so, can you make a repeated measures
comparison using your software?
In response to the first part of the question, yes; the SF-36v2™ Health Survey is often used in longitudinal studies to determine changes in a patient's health over time.
Results from previous administrations are reported on the individual and provider reports. However, the QualityMetric Health Outcomes™ Scoring Software 2.0 does not have
the capability to analyze data (t-test, regression analysis, ANOVA, etc.). You need a traditional data analytic software package (e.g. SPSS, SAS, STATA) to make the repeated measures comparisons.
What are the minimum system requirements to install the QualityMetric Health Outcomes™ Scoring Software 2.0?
Windows XP requirements:
-
Pentium 233 megahertz (MHz) processor or faster (300 MHz is recommended)
-
At least 64 megabytes (MB) of RAM (128 MB is recommended)
-
At least 100 megabytes (MB) of available space on the hard disk
-
CD-ROM or DVD-ROM drive
-
Video adapter and monitor with Super VGA (800 x 600)or higher resolution
Windows Vista requirements:
-
A modern processor (at least 800MHz1)
-
At least 100 megabytes (MB) of available space on the hard disk
-
512 MB of system memory
-
A graphics processor that is DirectX 9 capable
The application also requires that version 2.0 of the Microsoft .NET Framework and the Microsoft Installer (MSI)
version 3 be installed. Both are included in the installer.
Software requirements:
-
Supported Operating Systems:
-
Windows Vista
-
Windows Server 2003*
-
Windows XP**
*Requires the latest version of the Microsoft .NET Framework, which is included with
the software and is available for download from
www.microsoft.com.
**Requires Microsoft Internet Explorer 6.0 or higher.
I am running a Macintosh operating system. Can I install the QualityMetric Health Outcomes™ Scoring Software 2.0 on this system?
Macintosh is not a supported operating system for this software product. The installation and operation of the QualityMetric Health Outcomes™ Scoring Software 2.0 is only compatible with Microsoft operating systems.
What else do I need to know before installing the QualityMetric Health Outcomes™ Scoring Software 2.0 on my desktop?
Before you begin the installation procedure, you should:
-
Exit any programs currently running on your computer.
-
Have your Activation Key that you received from QualityMetric when you purchased the QualityMetric Health Outcomes™ Scoring Software 2.0 available. The Activation Key is printed on the software CD sleeve, and sent to you by email.
-
Make sure your computer system meets the minimum requirements.
Why do I need an activation key?
This is to verify that you are licensed to install and use the software on your computer. Your Activation Key consists of a series of alphanumeric character strings.
How does the QualityMetric Health Outcomes™ Scoring Software 2.0 score my raw data set?
Once you have identified the survey you used and your collected dataset has been entered into a data grid, formatted to a comma-separated variable (.CSV) file or
computer administered, you can now run the QualityMetric Health Outcomes™ Scoring Software 2.0.
If you are importing data, the QualityMetric Health Outcomes™ Scoring Software 2.0 requires that your raw data set be formatted in the following manner:
-
The file must be comma-separated text.
-
The first line in the file must contain the item labels for the health survey.
-
Subsequent lines in the file must contain raw (e.g. not recoded or reversed) response data.
-
Records may contain data other than item responses, such as patient identifiers.
-
Each line in the file must contain no more than one survey record.
What is a comma-separated variable (.CSV) file?
This is a file that consists of values in a table as a series of ASCII text lines that are organized so that each value in a column is separated by a comma from the next column value
and each row begins with a new line. The .CSV file can be compiled using any spreadsheet software such as Microsoft® Excel. To save an Excel spreadsheet in a comma-separated format,
follow these steps:
-
Click "File" on the Toolbar
-
Click "Save As" on the drop-down menu
-
In the "Save As Type" window, click the drop-down list
-
Scroll down to "CSV(comma delimited)(*.csv)," and select this to save the file
What is ASCII text?
The American Standard Code for Information Interchange (ASCII) was developed by the American National Standards Institute. It is a common format used across
the world as a standard for code numbers used by computers to represent all upper and lower case characters, such as letters, numbers, and punctuations.
I have inserted the QualityMetric Health Outcomes™ Scoring Software 2.0 CD into the CD-ROM drive and the application does not seem to be running the
installation process. What seems to be the problem?
If you encounter this problem, you will have to run the software setup manually on your desktop. Here are the steps to run the manual Installation:
-
Click on [Start] on your Windows screen
-
Select [Run] from the menu
-
Type the letter of your CD ROM drive (Usually "D") followed by a colon (:) and the file, "Setup.exe". Click the "OK" button. (Example: "D:Setup.exe")
-
The install screen will appear. Follow the on-screen instructions.
If you have followed the above steps and still have issues with installing your software, please contact QualityMetric's technical support at 1-800-572-9394.
Is there the facility to output data (and variable names, etc.) to SPSS or other statistical packages?
The QualityMetric Health Outcomes™ Scoring Software 2.0 outputs the data only to a comma separated (.CSV) ASCII text file. These .CSV files may be viewed in
Microsoft® Excel or any other spreadsheet application. There is a mechanism that will allow you to import these .CSV files into SPSS and/or SAS. You may find
additional information on how to do this at the following web sites:
For SPSS information:
http://swdb.berkeley.edu/data/import.html#spsscsv
For SAS and SPSS information:
http://cc.uoregon.edu/cnews/summer2003/sasupdate.html
I want to know the output options in order to integrate the scored data to other SAS data?
The QualityMetric Health Outcomes™ Scoring Software 2.0 only outputs the data to a comma separated (.CSV) ASCII text file. These .CSV files may be viewed in
Microsoft® Excel or any other spreadsheet application. There is a mechanism that will allow you to import these .CSV files into SPSS and/or SAS. You may find
additional information on how to do this at the following web sites:
For SPSS information:
http://swdb.berkeley.edu/data/import.html#spsscsv
For SAS and SPSS information:
http://cc.uoregon.edu/cnews/summer2003/sasupdate.html
Any additional manipulation and/or analysis of the data within SAS or other analytical software packages would need to be programmed by the user.
Does your software include a scanner to scan in results?
At this time, we do not offer any sort of scanning tools to import HRQOL data. However, the QualityMetric Health Outcomes™ Scoring Software 2.0 would still
be useful in scoring the data once it is imported from the scanned forms and is transformed into the appropriate format.
